The context: Why is this problem important to solve?
The objectives: What is the intended goal?
The key questions: What are the key questions that need to be answered?
The problem formulation: What are we trying to solve using data science?
⏩ Deep Learning has found applications in many predictive tasks relating to more unstructured forms of data over the last few years, such as images, text, audio and video. Many of these tasks seem to be in the vein of a larger direction of predictive modeling that aims to match human-level performance on such tasks, because humans have evolved to specialize in performing intelligent actions on such unstructured data. As a specific branch of AI (also called Affective Computing or Emotion AI) Artificial Emotional Intelligence stands for the study and development of technologies and computers that can read human emotions by means of analyzing body gestures, facial expressions, voice tone, etc. and react appropriately to them.
In the field of human-machine interaction, facial expression recognition is critical. From recent research, it has been found that as much as 55% of communication of sentiment takes place through facial expressions and other visual cues. Therefore, training a model to identify facial emotions accurately is an important step towards the development of emotionally intelligent behavior in machines with AI capabilities. Automatic facial expression recognition systems could have many applications, including but not limited to any use case that requires human behavior understanding, detection of mental disorders, and creating a higher quality of virtual assistant for customer-facing businesses.
⏩ The goal of this project is to use Deep Learning and Artificial Intelligence techniques to create a computer vision model that can accurately detect facial emotions. The model should be able to perform multi-class classification on images of facial expressions, to classify the expressions according to the associated emotion.
⏩ The data set consists of 3 folders, i.e., 'test', 'train', and 'validation'. Each of these folders has four subfolders:
‘happy’: Images of people who have happy facial expressions.
‘sad’: Images of people with sad or upset facial expressions.
‘surprise’: Images of people who have shocked or surprised facial expressions.
‘neutral’: Images of people showing no prominent emotion in their facial expression at all.
NOTE: Please use Google Colab from your browser for this notebook. Google.colab is NOT a library that can be downloaded locally on your device.
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
!ls '/content/drive/MyDrive/Colab Notebooks/Capstone/data/'
Facial_emotion_images.zip
# Importing data support libraries, including graph support
import math
import matplotlib.pyplot as plt
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np
import operator
import os
import pandas as pd
import pickle
import random
import seaborn as sns
import zipfile
# Importing Machine Learning utilities
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import f1_score
from sklearn.metrics import accuracy_score
# Importing Deep Learning Libraries
import tensorflow as tf
import tensorflow.keras.applications as ap
from tensorflow.keras import backend
from tensorflow.keras.layers import Activation
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Embedding
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import GlobalAveragePooling2D
from tensorflow.keras.layers import Input
from tensorflow.keras.layers import LeakyReLU
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Reshape
from tensorflow.keras.losses import categorical_crossentropy
from tensorflow.keras.models import Model
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.utils import to_categorical
from keras.applications.vgg16 import VGG16
from keras.callbacks import EarlyStopping
from keras.callbacks import ModelCheckpoint
from keras.callbacks import ReduceLROnPlateau
# For environment checklist
from psutil import virtual_memory
# Reviewing TensorFlow library version
print('TF version: ', tf.__version__)
try:
tpu = tf.distribute.cluster_resolver.TPUClusterResolver() # TPU detection
print('Running on TPU ', tpu.cluster_spec().as_dict()['worker'])
except:
print('ERROR: Not connected to a TPU runtime')
TF version: 2.12.0 Running on TPU ['10.30.87.242:8470']
# Reviewing the memory available
ram_gb = virtual_memory().total / 1e9
print('Runtime has {:.1f} gigabytes of available RAM'.format(ram_gb))
if ram_gb < 20:
print('Not using a high-RAM runtime')
else:
print('Using a high-RAM runtime!')
Runtime has 37.8 gigabytes of available RAM Using a high-RAM runtime!
Note:
file = '/content/drive/MyDrive/Colab Notebooks/Capstone/data/Facial_emotion_images.zip'
# The data is provided as a zip file so we need to extract it from the zip file
with zipfile.ZipFile(file, 'r') as zip_ref:
zip_ref.extractall()
# Global variables
IMG_DIR = '/content/Facial_emotion_images'
MODEL_DIR = f'/content/drive/MyDrive/Colab Notebooks/Capstone/model'
TRAIN_PATH = IMG_DIR + '/train'
TEST_PATH = IMG_DIR + '/test'
VAL_PATH = IMG_DIR + '/validation'
SEED = 0
FACE_EXPRESSIONS = ['happy', 'neutral', 'sad', 'surprise']
Let's look at our classes.
Write down your observation for each class. What do you think can be a unique feature of each emotion, that separates it from the remaining classes?
def reading_files(path, name_of_set, show_sample_images=True, pick_random=True):
"""Read files in the directory path.
It can shows random sample of images if it is desired by setting
show_sample_images and pick_random parameters.
"""
files_to_read = os.listdir(path)
print(f'Total files in {name_of_set}: {len(files_to_read)}')
if show_sample_images:
fig = plt.figure(figsize=(10, 2))
if pick_random:
images = [os.path.join(path,
files_to_read[random.randint(0, len(files_to_read)-1)])
for i in range(8)]
else:
images = [os.path.join(path, file) for file in sorted(files_to_read)[-8:]]
for i, image in enumerate(images, 1):
ax = plt.subplot(1, 8, i)
ax.axis('Off')
plt.imshow(load_img(image))
plt.title(image.split('/')[-1], fontsize=6)
fig.suptitle(name_of_set, fontsize=8)
plt.tight_layout()
plt.show()
return files_to_read
def read_and_display_images_in_set(path, set_name, show_sample_images=True, pick_random=True):
"""Read over different subfolders (categories) in the directory path.
"""
files_in_set = {}
expressions = FACE_EXPRESSIONS
feature = []
for expression in expressions:
files_path = f'{path}/{expression}'
name_of_set = f'{set_name} - {expression.upper()} faces'
files_in_set[expression] = reading_files(files_path, name_of_set, show_sample_images, pick_random)
feature += [expression] * len(files_in_set[expression])
print(f'Total images: {len(feature)}')
return files_in_set, pd.Series(feature)
def get_feature_countplot(feature, set_name):
"""Countplot of a categorical variable with quantity label on each catego.
"""
sns.set(font_scale=.75)
total = len(feature) # Length of the column
plt.figure(figsize=(10, 3))
# Convert the column to a categorical data type
feature = feature.astype('category')
origin = feature.copy()
labels = feature.value_counts().index
ax = sns.countplot(x=feature, palette='Paired', order=labels)
ax.set_xlabel('')
# custom label calculates percent and add an empty string so 0 value bars don't have a number
for container in ax.containers:
labels = [f'{h:.0f}\n( {h/origin.count()*100:0.1f}% )'
if (h := v.get_height()) > 0 else '' for v in container]
ax.bar_label(container, labels=labels, label_type='edge',
fontsize='small', weight='bold') # color='white', label_type='center'
ylim = plt.ylim()
plt.ylim(ylim[0], ylim[1]*1.1)
plt.title(f'Data Distribution in {set_name}')
plt.show()
plt.style.use('default')
Training Set
title_of_set = 'Train Set'
training_set, feature_train = read_and_display_images_in_set(TRAIN_PATH, title_of_set)
Total files in Train Set - HAPPY faces: 3976
Total files in Train Set - NEUTRAL faces: 3978
Total files in Train Set - SAD faces: 3982
Total files in Train Set - SURPRISE faces: 3173
Total images: 15109
get_feature_countplot(feature_train, title_of_set)
Testing Set
title_of_set = 'Test Set'
testing_set, feature_test = read_and_display_images_in_set(TEST_PATH, title_of_set, pick_random=False)
Total files in Test Set - HAPPY faces: 32
Total files in Test Set - NEUTRAL faces: 32
Total files in Test Set - SAD faces: 32
Total files in Test Set - SURPRISE faces: 32
Total images: 128
get_feature_countplot(feature_test, title_of_set)
Validation Set
title_of_set = 'Validation Set'
validation_set, feature_val = read_and_display_images_in_set(VAL_PATH, title_of_set)
Total files in Validation Set - HAPPY faces: 1825
Total files in Validation Set - NEUTRAL faces: 1216
Total files in Validation Set - SAD faces: 1139
Total files in Validation Set - SURPRISE faces: 797
Total images: 4977
get_feature_countplot(feature_val, title_of_set)
⏩ Observations and Insights:
fig = plt.figure(figsize=(6, 2))
ax = plt.subplot(1, 2, 1)
ax.axis('Off')
plt.imshow(load_img(f'{TRAIN_PATH}/neutral/32222.jpg'))
plt.title('Trainning Set \nNeutral \n32222.jpg', fontsize=9)
ax = plt.subplot(1, 2, 2)
ax.axis('Off')
plt.imshow(load_img(f'{VAL_PATH}/happy/9984.jpg'))
plt.title('Validation Set \nHappy \n9984.jpg', fontsize=9)
plt.tight_layout()
plt.show()
# Deleting the wrong images
!rm '/content/Facial_emotion_images/test/happy/15838.jpg'
# Recalculating the testing_set and feature_test
title_of_set = 'Test Set'
testing_set, feature_test = read_and_display_images_in_set(TEST_PATH, title_of_set, show_sample_images=False)
Total files in Test Set - HAPPY faces: 31 Total files in Test Set - NEUTRAL faces: 32 Total files in Test Set - SAD faces: 32 Total files in Test Set - SURPRISE faces: 32 Total images: 127
# Observing the dimenssions of images in each set
def display_dimmension_images_in_set(path, data_set, set_name):
"""Display a scatterplot of images dimmension in the set
"""
dim = []
for expression, images in data_set.items():
files_path = f'{path}/{expression}'
dim += [(img_to_array(load_img(f'{files_path}/{image}')).shape) for image in images]
if len(set(map(operator.itemgetter(-1), dim))) == 1:
print(f'All elements in "{expression}" category has {dim[0][-1]} dimmensions')
x = list(map(operator.itemgetter(0), dim))
y = list(map(operator.itemgetter(1), dim))
fig = plt.figure(figsize=(6, 2))
plt.scatter(x,y)
plt.title(set_name)
plt.show()
display_dimmension_images_in_set(TRAIN_PATH, training_set, 'Training Set')
All elements in "happy" category has 3 dimmensions All elements in "neutral" category has 3 dimmensions All elements in "sad" category has 3 dimmensions All elements in "surprise" category has 3 dimmensions
display_dimmension_images_in_set(TEST_PATH, testing_set, 'Testing Set')
All elements in "happy" category has 3 dimmensions All elements in "neutral" category has 3 dimmensions All elements in "sad" category has 3 dimmensions All elements in "surprise" category has 3 dimmensions
display_dimmension_images_in_set(VAL_PATH, validation_set, 'Validation Set')
All elements in "happy" category has 3 dimmensions All elements in "neutral" category has 3 dimmensions All elements in "sad" category has 3 dimmensions All elements in "surprise" category has 3 dimmensions
Think About It:
⏩ Observations and Insights:
In this section, we are creating data loaders that we will use as inputs to our Neural Network.
You have two options for the color_mode. You can set it to color_mode = 'rgb' or color_mode = 'grayscale'. You will need to try out both and see for yourself which one gives better performance.
def create_data_loaders(path, color_mode, batch_size=64):
"""Create the data loaders for training, testing and validation sets.
It will be necessary to definte the color mode: rgb or grayscale.
"""
# All images to be rescaled by 1/255.
datagen_train = ImageDataGenerator(horizontal_flip = True,
rescale=1./255,
shear_range=0.3)
datagen_test = ImageDataGenerator(rescale = 1.0/255.)
datagen_val = ImageDataGenerator(rescale = 1.0/255.)
# Generating the batches od the images
default_params = {
'target_size': (48, 48),
'color_mode' : color_mode,
'class_mode' : 'categorical',
'classes' : FACE_EXPRESSIONS,
'shuffle' : True
}
print(f'\nData with color_mode={color_mode}:')
train_generator = datagen_train.flow_from_directory(f'{path}/train', batch_size=batch_size, **default_params)
test_generator = datagen_train.flow_from_directory(f'{path}/test', batch_size=32, **default_params)
val_generator = datagen_train.flow_from_directory(f'{path}/validation', batch_size=batch_size, **default_params)
return train_generator, test_generator, val_generator
train_generator_rgb, _, _ = create_data_loaders(IMG_DIR, 'rgb')
# Taking a look at some examples of our augmented training data with color_mode='rgb'.
images, labels = next(train_generator_rgb)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image)
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=rgb: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
train_generator_gray, _, _ = create_data_loaders(IMG_DIR, 'grayscale')
# Taking a look at some examples of our augmented training data with color_mode='grayscale'.
images, labels = next(train_generator_gray)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image)
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
# Using proper cmap
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image, cmap='gray')
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=grayscale: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
def save_object(obj, filename):
"""To save objects that can be loaded later"""
with open(filename, 'wb') as f:
pickle.dump(obj, f, pickle.HIGHEST_PROTOCOL)
def metrics_score(actual, predicted, model_name, history):
"""Calculating metrics to later compare performance of the model.
It also prints the classification report and the confusion matrix
"""
print(classification_report(actual, predicted, zero_division=0, target_names=FACE_EXPRESSIONS))
cm = confusion_matrix(actual, predicted)
plt.figure(figsize = (8, 5))
sns.heatmap(cm, annot=True, fmt='.0f', xticklabels=FACE_EXPRESSIONS, yticklabels=FACE_EXPRESSIONS)
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.show()
# Creating a dataframe of metrics
index = np.argmax(history.history['val_accuracy'])
df = pd.DataFrame(
data={
"Accuracy Train": history.history['accuracy'][index],
"Accuracy Val": history.history['val_accuracy'][index],
"Accuracy Test": accuracy_score(actual, predicted)
},
index=[model_name],
)
return df
def plot_model_result(result, model_name):
""" Plot the accuracy and loss values of the model.
"""
fig = plt.figure(figsize = (10, 3))
# Plotting the Accuracy
plt.subplot(1, 2, 1)
plt.plot(result.history['accuracy'])
plt.plot(result.history['val_accuracy'])
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['train', 'validation'], loc='upper left')
# Plotting the Loss
plt.subplot(1, 2, 2)
plt.plot(result.history['loss'])
plt.plot(result.history['val_loss'])
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['train', 'validation'], loc='upper left')
fig.suptitle(model_name)
plt.show()
def init_environment():
# Clearing previous TF sessions
backend.clear_session()
# Fixing the seed for random number generators
os.environ['PYTHONHASHSEED'] = str(2)
np.random.seed(SEED)
random.seed(SEED)
tf.random.set_seed(SEED)
print('Initializing TF Session and Seed')
⏩ Steps to follow in the model building:
def model_building_and_evaluating_process(model_builder, model_base_name, batch_size=64,
include_grayscale=True, include_rgb=True, epochs=20, tune=0):
"""Execute the following steps: (1) Initialize the environment, (2) Build and compile the model based on the
model builder method, (3) Train the model on the trainning set, and (4) Evaluate the model on the testing set
"""
model_sets = []
if include_rgb:
model_sets.append(('RGB', (48, 48, 3)))
if include_grayscale:
model_sets.append(('GRAY', (48, 48, 1)))
model_metrics = pd.DataFrame({})
model_history = {}
fitted_model = {}
for img_set, input_shape in model_sets:
model_name = f'{model_base_name} - {img_set}'
print('---------------------------------------------')
print(model_name)
print('---------------------------------------------')
# Step 1: Initialize the environment
init_environment()
train_set, test_set, val_set = create_data_loaders(IMG_DIR, color_mode='rgb' if img_set=='RGB' else 'grayscale',
batch_size=batch_size)
X_test, y_test = test_set.next()
print('\n\n')
# Step 2: Build and compile the model
if tune:
model = model_builder(input_shape=input_shape, tune=tune)
else:
model = model_builder(input_shape=input_shape)
print('\n\n')
# Step 3: Train the model on the training set
checkpoint = ModelCheckpoint(f'{MODEL_DIR}/{model_name}.h5', monitor='val_accuracy', verbose=0, save_best_only=True, mode='max')
early_stopping = EarlyStopping(monitor='val_loss', min_delta=0, patience=5, verbose=0, restore_best_weights=True)
reduce_learningrate = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3, verbose=0, min_delta=0.0001)
history = model.fit(train_set,
validation_data=val_set,
epochs=epochs,
callbacks=[early_stopping, checkpoint, reduce_learningrate])
print('\n\n')
# Step 4: Evaluate the model on the testing set
model.evaluate(X_test, y_test, verbose=1)
y_pred = np.argmax(model.predict(X_test), axis=1)
y_test = np.argmax(y_test, axis=1)
print('Actual :', y_test)
print('Prediction:', y_pred)
print('\n\n')
results = metrics_score(y_test, y_pred, model_name, history)
model_metrics = pd.concat([model_metrics, results])
print('\n\n')
fitted_model[model_name] = model
model_history[model_name] = history
plot_model_result(history, model_name)
print('\n\n')
return fitted_model, model_metrics, model_history
def loss_and_accuracy_comparisson(model_history, data_type='Training and Validation Set'):
"""Visualizing the loss and the accuracy on the train and the
validation data for all executed models.
The "data_type" to plot could be "Training and Validation Set", "Validation Set", "Training Set".
"""
fig, (ax1, ax2) = plt.subplots(1, 2, figsize = (10, 3))
for model_name, history in model_history.items():
if data_type in ["Training and Validation Set", "Training Set"]:
ax1.plot(history.history['accuracy'], ls='-', lw=1, label=f'{model_name}')
if data_type in ["Training and Validation Set", "Validation Set"]:
ax1.plot(history.history['val_accuracy'], ls='--', lw=1, label=f'{model_name} val')
ax1.set_ylabel('Accuracy', fontsize=8)
ax1.set_xlabel('Epochs', fontsize=8)
ax1.legend(fontsize=6)
ax1.set_title('Accuracy Comparisson', fontsize=8)
ax1.xaxis.set_tick_params(labelsize=7)
ax1.yaxis.set_tick_params(labelsize=7)
if data_type in ["Training and Validation Set", "Training Set"]:
ax2.plot(history.history['loss'], ls='-', lw=1, label=f'{model_name}')
if data_type in ["Training and Validation Set", "Validation Set"]:
ax2.plot(history.history['val_loss'], ls='--', lw=1, label=f'{model_name} val')
ax2.set_ylabel('Loss', fontsize=8)
ax2.set_xlabel('Epochs', fontsize=8)
ax2.legend(fontsize=6)
ax2.set_title('Loss Comparisson', fontsize=8)
ax2.xaxis.set_tick_params(labelsize=7)
ax2.yaxis.set_tick_params(labelsize=7)
fig.suptitle(f'Model Comparisson - {data_type}', fontsize=10)
plt.tight_layout()
plt.show()
Think About It:
⏩ Insights:
# Model 1: Base Neural Network
def model_builder_base_cnn(input_shape):
"""Creating the Base Neural Network.
"""
model = Sequential([
Conv2D(filters=64, kernel_size=2, activation='relu', padding='same', input_shape=input_shape),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=32, kernel_size=2, activation='relu', padding='same'),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=32, kernel_size=2, activation='relu', padding='same'),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Flatten(),
Dense(512, activation='relu'),
Dropout(0.4),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_base_cnn, metrics_base_cnn, history_base_cnn = model_building_and_evaluating_process(
model_builder_base_cnn, f'Model 1: Base Neural Network ({batch_size})', batch_size=batch_size, epochs=20
)
---------------------------------------------
Model 1: Base Neural Network (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 832
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,572
Trainable params: 605,572
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 28s 56ms/step - loss: 1.3195 - accuracy: 0.3512 - val_loss: 1.1722 - val_accuracy: 0.5055 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 26s 55ms/step - loss: 1.1272 - accuracy: 0.5053 - val_loss: 1.0094 - val_accuracy: 0.5763 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 26s 55ms/step - loss: 1.0300 - accuracy: 0.5603 - val_loss: 0.9365 - val_accuracy: 0.6156 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 26s 55ms/step - loss: 0.9791 - accuracy: 0.5833 - val_loss: 0.8959 - val_accuracy: 0.6381 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 26s 54ms/step - loss: 0.9427 - accuracy: 0.6008 - val_loss: 0.8576 - val_accuracy: 0.6426 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 26s 55ms/step - loss: 0.9150 - accuracy: 0.6117 - val_loss: 0.8524 - val_accuracy: 0.6460 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 26s 54ms/step - loss: 0.8823 - accuracy: 0.6273 - val_loss: 0.8281 - val_accuracy: 0.6558 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 26s 54ms/step - loss: 0.8707 - accuracy: 0.6345 - val_loss: 0.8111 - val_accuracy: 0.6628 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 25s 53ms/step - loss: 0.8465 - accuracy: 0.6435 - val_loss: 0.9083 - val_accuracy: 0.6211 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 26s 54ms/step - loss: 0.8320 - accuracy: 0.6503 - val_loss: 0.7770 - val_accuracy: 0.6815 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 25s 54ms/step - loss: 0.8169 - accuracy: 0.6613 - val_loss: 0.7744 - val_accuracy: 0.6795 - lr: 0.0010
Epoch 12/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7962 - accuracy: 0.6653 - val_loss: 0.7571 - val_accuracy: 0.6908 - lr: 0.0010
Epoch 13/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7925 - accuracy: 0.6695 - val_loss: 0.7417 - val_accuracy: 0.6934 - lr: 0.0010
Epoch 14/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7785 - accuracy: 0.6754 - val_loss: 0.7430 - val_accuracy: 0.6952 - lr: 0.0010
Epoch 15/20
473/473 [==============================] - 25s 53ms/step - loss: 0.7664 - accuracy: 0.6851 - val_loss: 0.7474 - val_accuracy: 0.6946 - lr: 0.0010
Epoch 16/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7591 - accuracy: 0.6842 - val_loss: 0.7310 - val_accuracy: 0.7103 - lr: 0.0010
Epoch 17/20
473/473 [==============================] - 25s 53ms/step - loss: 0.7407 - accuracy: 0.6998 - val_loss: 0.7458 - val_accuracy: 0.6922 - lr: 0.0010
Epoch 18/20
473/473 [==============================] - 25s 54ms/step - loss: 0.7391 - accuracy: 0.6977 - val_loss: 0.7134 - val_accuracy: 0.7153 - lr: 0.0010
Epoch 19/20
473/473 [==============================] - 25s 53ms/step - loss: 0.7329 - accuracy: 0.6989 - val_loss: 0.7137 - val_accuracy: 0.7137 - lr: 0.0010
Epoch 20/20
473/473 [==============================] - 26s 54ms/step - loss: 0.7171 - accuracy: 0.7059 - val_loss: 0.7141 - val_accuracy: 0.7191 - lr: 0.0010
1/1 [==============================] - 0s 33ms/step - loss: 0.6004 - accuracy: 0.7812
1/1 [==============================] - 0s 111ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 1 3 2 2 3 0 3 2 0 2 2 3 2 2 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.80 0.89 10
neutral 0.43 0.60 0.50 5
sad 0.73 0.80 0.76 10
surprise 1.00 0.86 0.92 7
accuracy 0.78 32
macro avg 0.79 0.76 0.77 32
weighted avg 0.83 0.78 0.80 32
---------------------------------------------
Model 1: Base Neural Network (32) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,060
Trainable params: 605,060
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 18s 36ms/step - loss: 1.2927 - accuracy: 0.3759 - val_loss: 1.1302 - val_accuracy: 0.5214 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 17s 35ms/step - loss: 1.1190 - accuracy: 0.5128 - val_loss: 1.0353 - val_accuracy: 0.5712 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 17s 35ms/step - loss: 1.0362 - accuracy: 0.5545 - val_loss: 0.9599 - val_accuracy: 0.5931 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 17s 35ms/step - loss: 0.9930 - accuracy: 0.5734 - val_loss: 0.9279 - val_accuracy: 0.6078 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 17s 36ms/step - loss: 0.9548 - accuracy: 0.5902 - val_loss: 0.8821 - val_accuracy: 0.6331 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 17s 35ms/step - loss: 0.9330 - accuracy: 0.6047 - val_loss: 0.8614 - val_accuracy: 0.6458 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 17s 35ms/step - loss: 0.9047 - accuracy: 0.6176 - val_loss: 0.8477 - val_accuracy: 0.6552 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 17s 36ms/step - loss: 0.8879 - accuracy: 0.6253 - val_loss: 0.8345 - val_accuracy: 0.6566 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 16s 35ms/step - loss: 0.8648 - accuracy: 0.6349 - val_loss: 0.8628 - val_accuracy: 0.6510 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 17s 35ms/step - loss: 0.8520 - accuracy: 0.6444 - val_loss: 0.7941 - val_accuracy: 0.6789 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 16s 35ms/step - loss: 0.8362 - accuracy: 0.6494 - val_loss: 0.7748 - val_accuracy: 0.6781 - lr: 0.0010
Epoch 12/20
473/473 [==============================] - 17s 35ms/step - loss: 0.8236 - accuracy: 0.6538 - val_loss: 0.7743 - val_accuracy: 0.6805 - lr: 0.0010
Epoch 13/20
473/473 [==============================] - 17s 35ms/step - loss: 0.8113 - accuracy: 0.6628 - val_loss: 0.7691 - val_accuracy: 0.6827 - lr: 0.0010
Epoch 14/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7949 - accuracy: 0.6744 - val_loss: 0.7791 - val_accuracy: 0.6783 - lr: 0.0010
Epoch 15/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7895 - accuracy: 0.6734 - val_loss: 0.7701 - val_accuracy: 0.6821 - lr: 0.0010
Epoch 16/20
473/473 [==============================] - 16s 35ms/step - loss: 0.7774 - accuracy: 0.6793 - val_loss: 0.7605 - val_accuracy: 0.6960 - lr: 0.0010
Epoch 17/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7618 - accuracy: 0.6881 - val_loss: 0.7565 - val_accuracy: 0.6890 - lr: 0.0010
Epoch 18/20
473/473 [==============================] - 16s 35ms/step - loss: 0.7623 - accuracy: 0.6848 - val_loss: 0.7480 - val_accuracy: 0.7022 - lr: 0.0010
Epoch 19/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7532 - accuracy: 0.6906 - val_loss: 0.7451 - val_accuracy: 0.7010 - lr: 0.0010
Epoch 20/20
473/473 [==============================] - 16s 34ms/step - loss: 0.7371 - accuracy: 0.6977 - val_loss: 0.7562 - val_accuracy: 0.6924 - lr: 0.0010
1/1 [==============================] - 0s 27ms/step - loss: 0.7368 - accuracy: 0.7812
1/1 [==============================] - 0s 81ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 1 3 2 1 0 2 1 3 2 2 3 0 3 2 0 2 2 2 1 2 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.80 0.89 10
neutral 0.44 0.80 0.57 5
sad 0.80 0.80 0.80 10
surprise 1.00 0.71 0.83 7
accuracy 0.78 32
macro avg 0.81 0.78 0.77 32
weighted avg 0.85 0.78 0.80 32
batch_size = 64
model, results, history = model_building_and_evaluating_process(model_builder_base_cnn, f'Model 1: Base Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_base_cnn.update(model)
metrics_base_cnn = pd.concat([metrics_base_cnn, results])
history_base_cnn.update(history)
---------------------------------------------
Model 1: Base Neural Network (64) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 832
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,572
Trainable params: 605,572
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 26s 105ms/step - loss: 1.3350 - accuracy: 0.3336 - val_loss: 1.2285 - val_accuracy: 0.4406 - lr: 0.0010
Epoch 2/20
237/237 [==============================] - 25s 104ms/step - loss: 1.1514 - accuracy: 0.4954 - val_loss: 1.0440 - val_accuracy: 0.5630 - lr: 0.0010
Epoch 3/20
237/237 [==============================] - 25s 104ms/step - loss: 1.0589 - accuracy: 0.5458 - val_loss: 0.9748 - val_accuracy: 0.5929 - lr: 0.0010
Epoch 4/20
237/237 [==============================] - 25s 104ms/step - loss: 1.0048 - accuracy: 0.5693 - val_loss: 0.9210 - val_accuracy: 0.6174 - lr: 0.0010
Epoch 5/20
237/237 [==============================] - 25s 104ms/step - loss: 0.9633 - accuracy: 0.5898 - val_loss: 0.8824 - val_accuracy: 0.6305 - lr: 0.0010
Epoch 6/20
237/237 [==============================] - 25s 105ms/step - loss: 0.9483 - accuracy: 0.5953 - val_loss: 0.8572 - val_accuracy: 0.6438 - lr: 0.0010
Epoch 7/20
237/237 [==============================] - 25s 104ms/step - loss: 0.9134 - accuracy: 0.6138 - val_loss: 0.8352 - val_accuracy: 0.6582 - lr: 0.0010
Epoch 8/20
237/237 [==============================] - 24s 102ms/step - loss: 0.8938 - accuracy: 0.6241 - val_loss: 0.8455 - val_accuracy: 0.6524 - lr: 0.0010
Epoch 9/20
237/237 [==============================] - 25s 104ms/step - loss: 0.8755 - accuracy: 0.6327 - val_loss: 0.8110 - val_accuracy: 0.6707 - lr: 0.0010
Epoch 10/20
237/237 [==============================] - 24s 103ms/step - loss: 0.8609 - accuracy: 0.6434 - val_loss: 0.8108 - val_accuracy: 0.6631 - lr: 0.0010
Epoch 11/20
237/237 [==============================] - 24s 103ms/step - loss: 0.8399 - accuracy: 0.6482 - val_loss: 0.8028 - val_accuracy: 0.6705 - lr: 0.0010
Epoch 12/20
237/237 [==============================] - 25s 104ms/step - loss: 0.8305 - accuracy: 0.6559 - val_loss: 0.7610 - val_accuracy: 0.6892 - lr: 0.0010
Epoch 13/20
237/237 [==============================] - 24s 102ms/step - loss: 0.8115 - accuracy: 0.6662 - val_loss: 0.7873 - val_accuracy: 0.6729 - lr: 0.0010
Epoch 14/20
237/237 [==============================] - 25s 104ms/step - loss: 0.8060 - accuracy: 0.6654 - val_loss: 0.7576 - val_accuracy: 0.6902 - lr: 0.0010
Epoch 15/20
237/237 [==============================] - 25s 104ms/step - loss: 0.7871 - accuracy: 0.6735 - val_loss: 0.7419 - val_accuracy: 0.6998 - lr: 0.0010
Epoch 16/20
237/237 [==============================] - 24s 103ms/step - loss: 0.7744 - accuracy: 0.6811 - val_loss: 0.7882 - val_accuracy: 0.6783 - lr: 0.0010
Epoch 17/20
237/237 [==============================] - 24s 102ms/step - loss: 0.7680 - accuracy: 0.6866 - val_loss: 0.7927 - val_accuracy: 0.6757 - lr: 0.0010
Epoch 18/20
237/237 [==============================] - 24s 102ms/step - loss: 0.7544 - accuracy: 0.6879 - val_loss: 0.7602 - val_accuracy: 0.6854 - lr: 0.0010
Epoch 19/20
237/237 [==============================] - 25s 104ms/step - loss: 0.7134 - accuracy: 0.7036 - val_loss: 0.7214 - val_accuracy: 0.7054 - lr: 2.0000e-04
Epoch 20/20
237/237 [==============================] - 25s 103ms/step - loss: 0.7070 - accuracy: 0.7113 - val_loss: 0.7120 - val_accuracy: 0.7133 - lr: 2.0000e-04
1/1 [==============================] - 0s 30ms/step - loss: 0.6727 - accuracy: 0.7188
1/1 [==============================] - 0s 391ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 2 3 2 0 3 0 3 2 0 2 2 3 1 2 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.38 0.60 0.46 5
sad 0.67 0.60 0.63 10
surprise 1.00 0.86 0.92 7
accuracy 0.72 32
macro avg 0.73 0.71 0.71 32
weighted avg 0.76 0.72 0.73 32
---------------------------------------------
Model 1: Base Neural Network (64) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,060
Trainable params: 605,060
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 17s 68ms/step - loss: 1.3277 - accuracy: 0.3409 - val_loss: 1.1836 - val_accuracy: 0.4762 - lr: 0.0010
Epoch 2/20
237/237 [==============================] - 16s 67ms/step - loss: 1.1535 - accuracy: 0.4954 - val_loss: 1.0556 - val_accuracy: 0.5644 - lr: 0.0010
Epoch 3/20
237/237 [==============================] - 16s 67ms/step - loss: 1.0683 - accuracy: 0.5415 - val_loss: 0.9965 - val_accuracy: 0.5857 - lr: 0.0010
Epoch 4/20
237/237 [==============================] - 16s 67ms/step - loss: 1.0169 - accuracy: 0.5640 - val_loss: 0.9461 - val_accuracy: 0.6102 - lr: 0.0010
Epoch 5/20
237/237 [==============================] - 16s 67ms/step - loss: 0.9744 - accuracy: 0.5848 - val_loss: 0.9253 - val_accuracy: 0.6225 - lr: 0.0010
Epoch 6/20
237/237 [==============================] - 16s 67ms/step - loss: 0.9466 - accuracy: 0.5993 - val_loss: 0.8794 - val_accuracy: 0.6303 - lr: 0.0010
Epoch 7/20
237/237 [==============================] - 16s 67ms/step - loss: 0.9149 - accuracy: 0.6150 - val_loss: 0.8528 - val_accuracy: 0.6522 - lr: 0.0010
Epoch 8/20
237/237 [==============================] - 16s 66ms/step - loss: 0.8908 - accuracy: 0.6288 - val_loss: 0.8313 - val_accuracy: 0.6532 - lr: 0.0010
Epoch 9/20
237/237 [==============================] - 16s 67ms/step - loss: 0.8729 - accuracy: 0.6351 - val_loss: 0.8082 - val_accuracy: 0.6683 - lr: 0.0010
Epoch 10/20
237/237 [==============================] - 16s 65ms/step - loss: 0.8685 - accuracy: 0.6331 - val_loss: 0.8210 - val_accuracy: 0.6546 - lr: 0.0010
Epoch 11/20
237/237 [==============================] - 16s 65ms/step - loss: 0.8466 - accuracy: 0.6473 - val_loss: 0.8030 - val_accuracy: 0.6631 - lr: 0.0010
Epoch 12/20
237/237 [==============================] - 16s 68ms/step - loss: 0.8272 - accuracy: 0.6552 - val_loss: 0.7729 - val_accuracy: 0.6868 - lr: 0.0010
Epoch 13/20
237/237 [==============================] - 16s 65ms/step - loss: 0.8248 - accuracy: 0.6583 - val_loss: 0.8256 - val_accuracy: 0.6578 - lr: 0.0010
Epoch 14/20
237/237 [==============================] - 16s 67ms/step - loss: 0.8069 - accuracy: 0.6639 - val_loss: 0.7737 - val_accuracy: 0.6878 - lr: 0.0010
Epoch 15/20
237/237 [==============================] - 16s 67ms/step - loss: 0.7949 - accuracy: 0.6746 - val_loss: 0.7562 - val_accuracy: 0.6928 - lr: 0.0010
Epoch 16/20
237/237 [==============================] - 15s 65ms/step - loss: 0.7805 - accuracy: 0.6791 - val_loss: 0.7726 - val_accuracy: 0.6809 - lr: 0.0010
Epoch 17/20
237/237 [==============================] - 15s 65ms/step - loss: 0.7762 - accuracy: 0.6820 - val_loss: 0.7845 - val_accuracy: 0.6767 - lr: 0.0010
Epoch 18/20
237/237 [==============================] - 16s 65ms/step - loss: 0.7628 - accuracy: 0.6854 - val_loss: 0.7865 - val_accuracy: 0.6735 - lr: 0.0010
Epoch 19/20
237/237 [==============================] - 16s 67ms/step - loss: 0.7250 - accuracy: 0.7030 - val_loss: 0.7362 - val_accuracy: 0.7018 - lr: 2.0000e-04
Epoch 20/20
237/237 [==============================] - 16s 67ms/step - loss: 0.7234 - accuracy: 0.7088 - val_loss: 0.7283 - val_accuracy: 0.7038 - lr: 2.0000e-04
1/1 [==============================] - 0s 28ms/step - loss: 0.7231 - accuracy: 0.6250
1/1 [==============================] - 0s 84ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 0 0 2 3 2 1 0 1 2 3 2 0 3 0 3 2 0 2 2 2 1 1 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.80 0.80 0.80 10
neutral 0.38 0.60 0.46 5
sad 0.50 0.50 0.50 10
surprise 1.00 0.57 0.73 7
accuracy 0.62 32
macro avg 0.67 0.62 0.62 32
weighted avg 0.68 0.62 0.64 32
batch_size = 128
model, results, history = model_building_and_evaluating_process(model_builder_base_cnn, f'Model 1: Base Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_base_cnn.update(model)
metrics_base_cnn = pd.concat([metrics_base_cnn, results])
history_base_cnn.update(history)
---------------------------------------------
Model 1: Base Neural Network (128) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 832
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,572
Trainable params: 605,572
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 26s 207ms/step - loss: 1.3598 - accuracy: 0.3080 - val_loss: 1.3520 - val_accuracy: 0.2773 - lr: 0.0010
Epoch 2/20
119/119 [==============================] - 24s 204ms/step - loss: 1.2317 - accuracy: 0.4331 - val_loss: 1.1257 - val_accuracy: 0.5280 - lr: 0.0010
Epoch 3/20
119/119 [==============================] - 24s 204ms/step - loss: 1.1147 - accuracy: 0.5164 - val_loss: 1.0227 - val_accuracy: 0.5728 - lr: 0.0010
Epoch 4/20
119/119 [==============================] - 24s 202ms/step - loss: 1.0479 - accuracy: 0.5464 - val_loss: 0.9704 - val_accuracy: 0.6014 - lr: 0.0010
Epoch 5/20
119/119 [==============================] - 24s 203ms/step - loss: 1.0082 - accuracy: 0.5658 - val_loss: 0.9355 - val_accuracy: 0.6180 - lr: 0.0010
Epoch 6/20
119/119 [==============================] - 24s 205ms/step - loss: 0.9609 - accuracy: 0.5912 - val_loss: 0.9181 - val_accuracy: 0.6217 - lr: 0.0010
Epoch 7/20
119/119 [==============================] - 24s 205ms/step - loss: 0.9404 - accuracy: 0.6039 - val_loss: 0.8763 - val_accuracy: 0.6371 - lr: 0.0010
Epoch 8/20
119/119 [==============================] - 24s 203ms/step - loss: 0.9191 - accuracy: 0.6122 - val_loss: 0.8453 - val_accuracy: 0.6560 - lr: 0.0010
Epoch 9/20
119/119 [==============================] - 24s 200ms/step - loss: 0.9035 - accuracy: 0.6227 - val_loss: 0.8501 - val_accuracy: 0.6540 - lr: 0.0010
Epoch 10/20
119/119 [==============================] - 24s 200ms/step - loss: 0.8799 - accuracy: 0.6290 - val_loss: 0.8724 - val_accuracy: 0.6391 - lr: 0.0010
Epoch 11/20
119/119 [==============================] - 24s 203ms/step - loss: 0.8736 - accuracy: 0.6312 - val_loss: 0.8216 - val_accuracy: 0.6739 - lr: 0.0010
Epoch 12/20
119/119 [==============================] - 24s 204ms/step - loss: 0.8619 - accuracy: 0.6380 - val_loss: 0.8065 - val_accuracy: 0.6805 - lr: 0.0010
Epoch 13/20
119/119 [==============================] - 24s 200ms/step - loss: 0.8513 - accuracy: 0.6451 - val_loss: 0.7790 - val_accuracy: 0.6805 - lr: 0.0010
Epoch 14/20
119/119 [==============================] - 24s 203ms/step - loss: 0.8334 - accuracy: 0.6513 - val_loss: 0.7787 - val_accuracy: 0.6835 - lr: 0.0010
Epoch 15/20
119/119 [==============================] - 24s 205ms/step - loss: 0.8248 - accuracy: 0.6558 - val_loss: 0.7671 - val_accuracy: 0.6928 - lr: 0.0010
Epoch 16/20
119/119 [==============================] - 24s 200ms/step - loss: 0.8166 - accuracy: 0.6628 - val_loss: 0.7877 - val_accuracy: 0.6890 - lr: 0.0010
Epoch 17/20
119/119 [==============================] - 24s 202ms/step - loss: 0.8073 - accuracy: 0.6650 - val_loss: 0.7626 - val_accuracy: 0.6952 - lr: 0.0010
Epoch 18/20
119/119 [==============================] - 24s 201ms/step - loss: 0.7965 - accuracy: 0.6692 - val_loss: 0.7757 - val_accuracy: 0.6870 - lr: 0.0010
Epoch 19/20
119/119 [==============================] - 24s 204ms/step - loss: 0.7800 - accuracy: 0.6832 - val_loss: 0.7544 - val_accuracy: 0.7006 - lr: 0.0010
Epoch 20/20
119/119 [==============================] - 24s 206ms/step - loss: 0.7760 - accuracy: 0.6832 - val_loss: 0.7409 - val_accuracy: 0.7018 - lr: 0.0010
1/1 [==============================] - 0s 27ms/step - loss: 0.6682 - accuracy: 0.7500
WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fcaa01143a0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
1/1 [==============================] - 0s 86ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 0 3 2 1 0 1 2 3 2 2 3 0 3 2 0 2 2 3 1 2 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.43 0.60 0.50 5
sad 0.70 0.70 0.70 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.75 0.74 0.74 32
weighted avg 0.78 0.75 0.76 32
---------------------------------------------
Model 1: Base Neural Network (128) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 32) 8224
max_pooling2d_1 (MaxPooling (None, 12, 12, 32) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 32) 0
conv2d_2 (Conv2D) (None, 12, 12, 32) 4128
max_pooling2d_2 (MaxPooling (None, 6, 6, 32) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 32) 0
flatten (Flatten) (None, 1152) 0
dense (Dense) (None, 512) 590336
dropout_3 (Dropout) (None, 512) 0
dense_1 (Dense) (None, 4) 2052
=================================================================
Total params: 605,060
Trainable params: 605,060
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 17s 134ms/step - loss: 1.3531 - accuracy: 0.3193 - val_loss: 1.3037 - val_accuracy: 0.3824 - lr: 0.0010
Epoch 2/20
119/119 [==============================] - 16s 132ms/step - loss: 1.2397 - accuracy: 0.4326 - val_loss: 1.1524 - val_accuracy: 0.5101 - lr: 0.0010
Epoch 3/20
119/119 [==============================] - 16s 132ms/step - loss: 1.1440 - accuracy: 0.5014 - val_loss: 1.0613 - val_accuracy: 0.5634 - lr: 0.0010
Epoch 4/20
119/119 [==============================] - 16s 131ms/step - loss: 1.0848 - accuracy: 0.5296 - val_loss: 1.0169 - val_accuracy: 0.5813 - lr: 0.0010
Epoch 5/20
119/119 [==============================] - 16s 131ms/step - loss: 1.0316 - accuracy: 0.5626 - val_loss: 0.9673 - val_accuracy: 0.5998 - lr: 0.0010
Epoch 6/20
119/119 [==============================] - 16s 132ms/step - loss: 0.9936 - accuracy: 0.5797 - val_loss: 0.9314 - val_accuracy: 0.6148 - lr: 0.0010
Epoch 7/20
119/119 [==============================] - 16s 132ms/step - loss: 0.9713 - accuracy: 0.5880 - val_loss: 0.9358 - val_accuracy: 0.6170 - lr: 0.0010
Epoch 8/20
119/119 [==============================] - 16s 131ms/step - loss: 0.9474 - accuracy: 0.6008 - val_loss: 0.8824 - val_accuracy: 0.6432 - lr: 0.0010
Epoch 9/20
119/119 [==============================] - 16s 131ms/step - loss: 0.9232 - accuracy: 0.6106 - val_loss: 0.8665 - val_accuracy: 0.6462 - lr: 0.0010
Epoch 10/20
119/119 [==============================] - 15s 129ms/step - loss: 0.9045 - accuracy: 0.6182 - val_loss: 0.9163 - val_accuracy: 0.6074 - lr: 0.0010
Epoch 11/20
119/119 [==============================] - 16s 131ms/step - loss: 0.9012 - accuracy: 0.6183 - val_loss: 0.8631 - val_accuracy: 0.6476 - lr: 0.0010
Epoch 12/20
119/119 [==============================] - 16s 131ms/step - loss: 0.8793 - accuracy: 0.6354 - val_loss: 0.8176 - val_accuracy: 0.6679 - lr: 0.0010
Epoch 13/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8718 - accuracy: 0.6347 - val_loss: 0.8207 - val_accuracy: 0.6635 - lr: 0.0010
Epoch 14/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8559 - accuracy: 0.6441 - val_loss: 0.8093 - val_accuracy: 0.6667 - lr: 0.0010
Epoch 15/20
119/119 [==============================] - 16s 131ms/step - loss: 0.8443 - accuracy: 0.6516 - val_loss: 0.8021 - val_accuracy: 0.6733 - lr: 0.0010
Epoch 16/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8346 - accuracy: 0.6558 - val_loss: 0.8442 - val_accuracy: 0.6586 - lr: 0.0010
Epoch 17/20
119/119 [==============================] - 15s 127ms/step - loss: 0.8406 - accuracy: 0.6487 - val_loss: 0.8018 - val_accuracy: 0.6679 - lr: 0.0010
Epoch 18/20
119/119 [==============================] - 16s 130ms/step - loss: 0.8326 - accuracy: 0.6560 - val_loss: 0.7731 - val_accuracy: 0.6866 - lr: 0.0010
Epoch 19/20
119/119 [==============================] - 15s 128ms/step - loss: 0.8094 - accuracy: 0.6638 - val_loss: 0.7811 - val_accuracy: 0.6839 - lr: 0.0010
Epoch 20/20
119/119 [==============================] - 16s 130ms/step - loss: 0.8025 - accuracy: 0.6681 - val_loss: 0.7662 - val_accuracy: 0.6868 - lr: 0.0010
1/1 [==============================] - 0s 28ms/step - loss: 0.7159 - accuracy: 0.6875
WARNING:tensorflow:6 out of the last 6 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fca8027acb0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
1/1 [==============================] - 0s 87ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 0 3 0 3 2 2 2 2 3 1 2 0 2 1 0 3 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.33 0.40 0.36 5
sad 0.60 0.60 0.60 10
surprise 0.86 0.86 0.86 7
accuracy 0.69 32
macro avg 0.67 0.66 0.67 32
weighted avg 0.70 0.69 0.69 32
model_name = 'Model 1: Base Neural Network'
save_object(history_base_cnn, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metrics_base_cnn, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_base_cnn, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_base_cnn, data_type='Training Set')
loss_and_accuracy_comparisson(history_base_cnn, data_type='Validation Set')
metrics_base_cnn.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 1: Base Neural Network (32) - RGB | 0.705871 | 0.719108 | 0.781250 |
| Model 1: Base Neural Network (32) - GRAY | 0.684824 | 0.702230 | 0.781250 |
| Model 1: Base Neural Network (64) - RGB | 0.711298 | 0.713281 | 0.718750 |
| Model 1: Base Neural Network (64) - GRAY | 0.708783 | 0.703838 | 0.625000 |
| Model 1: Base Neural Network (128) - RGB | 0.683235 | 0.701828 | 0.750000 |
| Model 1: Base Neural Network (128) - GRAY | 0.668145 | 0.686759 | 0.687500 |
⏩ Observations and Insights:
⏩ Recommendations:
# Model 2: Enhanced Neural Network
def model_builder_enhanced_cnn(input_shape):
"""Creating an Enhanced Neural Network.
"""
model = Sequential([
Conv2D(filters=256, kernel_size=2, activation='relu', padding='same', input_shape=input_shape),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Conv2D(filters=128, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Conv2D(filters=64, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Conv2D(filters=32, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Flatten(),
Dense(512, activation='relu'),
Dense(128, activation='relu'),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.01) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_enhanced_cnn, metrics_enhanced_cnn, history_enhanced_cnn = model_building_and_evaluating_process(
model_builder_enhanced_cnn, f'Model 2: Enhanced Neural Network ({batch_size})', batch_size=batch_size, epochs=20
)
---------------------------------------------
Model 2: Enhanced Neural Network (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 3328
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 391,652
Trainable params: 390,692
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 69s 141ms/step - loss: 1.3893 - accuracy: 0.3587 - val_loss: 1.3292 - val_accuracy: 0.3757 - lr: 0.0100
Epoch 2/20
473/473 [==============================] - 66s 139ms/step - loss: 1.0473 - accuracy: 0.5200 - val_loss: 1.4469 - val_accuracy: 0.4061 - lr: 0.0100
Epoch 3/20
473/473 [==============================] - 66s 139ms/step - loss: 0.9427 - accuracy: 0.5763 - val_loss: 1.3244 - val_accuracy: 0.4541 - lr: 0.0100
Epoch 4/20
473/473 [==============================] - 66s 140ms/step - loss: 0.8756 - accuracy: 0.6178 - val_loss: 0.8860 - val_accuracy: 0.6295 - lr: 0.0100
Epoch 5/20
473/473 [==============================] - 66s 138ms/step - loss: 0.8234 - accuracy: 0.6549 - val_loss: 0.9004 - val_accuracy: 0.6227 - lr: 0.0100
Epoch 6/20
473/473 [==============================] - 66s 140ms/step - loss: 0.7941 - accuracy: 0.6711 - val_loss: 0.8781 - val_accuracy: 0.6309 - lr: 0.0100
Epoch 7/20
473/473 [==============================] - 66s 140ms/step - loss: 0.7566 - accuracy: 0.6875 - val_loss: 1.0392 - val_accuracy: 0.5425 - lr: 0.0100
Epoch 8/20
473/473 [==============================] - 67s 142ms/step - loss: 0.7365 - accuracy: 0.6935 - val_loss: 0.8052 - val_accuracy: 0.6683 - lr: 0.0100
Epoch 9/20
473/473 [==============================] - 66s 139ms/step - loss: 0.7253 - accuracy: 0.7033 - val_loss: 0.9822 - val_accuracy: 0.5943 - lr: 0.0100
Epoch 10/20
473/473 [==============================] - 66s 140ms/step - loss: 0.6975 - accuracy: 0.7157 - val_loss: 0.7527 - val_accuracy: 0.6962 - lr: 0.0100
Epoch 11/20
473/473 [==============================] - 65s 138ms/step - loss: 0.6905 - accuracy: 0.7151 - val_loss: 1.0202 - val_accuracy: 0.5959 - lr: 0.0100
Epoch 12/20
473/473 [==============================] - 65s 138ms/step - loss: 0.6701 - accuracy: 0.7220 - val_loss: 0.7829 - val_accuracy: 0.6759 - lr: 0.0100
Epoch 13/20
473/473 [==============================] - 66s 139ms/step - loss: 0.6630 - accuracy: 0.7287 - val_loss: 0.8674 - val_accuracy: 0.6582 - lr: 0.0100
Epoch 14/20
473/473 [==============================] - 66s 140ms/step - loss: 0.5698 - accuracy: 0.7699 - val_loss: 0.7297 - val_accuracy: 0.7215 - lr: 0.0020
Epoch 15/20
473/473 [==============================] - 66s 139ms/step - loss: 0.5381 - accuracy: 0.7824 - val_loss: 0.7504 - val_accuracy: 0.7131 - lr: 0.0020
Epoch 16/20
473/473 [==============================] - 66s 140ms/step - loss: 0.5154 - accuracy: 0.7923 - val_loss: 0.7193 - val_accuracy: 0.7271 - lr: 0.0020
Epoch 17/20
473/473 [==============================] - 67s 142ms/step - loss: 0.4999 - accuracy: 0.7931 - val_loss: 0.7519 - val_accuracy: 0.7183 - lr: 0.0020
Epoch 18/20
473/473 [==============================] - 80s 170ms/step - loss: 0.4833 - accuracy: 0.8048 - val_loss: 0.7527 - val_accuracy: 0.7189 - lr: 0.0020
Epoch 19/20
473/473 [==============================] - 79s 166ms/step - loss: 0.4647 - accuracy: 0.8126 - val_loss: 0.7593 - val_accuracy: 0.7265 - lr: 0.0020
Epoch 20/20
473/473 [==============================] - 78s 165ms/step - loss: 0.4299 - accuracy: 0.8293 - val_loss: 0.7631 - val_accuracy: 0.7255 - lr: 4.0000e-04
1/1 [==============================] - 0s 68ms/step - loss: 0.4700 - accuracy: 0.8438
1/1 [==============================] - 0s 180ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 1 2 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.90 0.95 10
neutral 0.57 0.80 0.67 5
sad 0.80 0.80 0.80 10
surprise 1.00 0.86 0.92 7
accuracy 0.84 32
macro avg 0.84 0.84 0.83 32
weighted avg 0.87 0.84 0.85 32
---------------------------------------------
Model 2: Enhanced Neural Network (32) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 1280
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 389,604
Trainable params: 388,644
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 78s 160ms/step - loss: 1.3877 - accuracy: 0.3525 - val_loss: 1.2700 - val_accuracy: 0.3434 - lr: 0.0100
Epoch 2/20
473/473 [==============================] - 72s 153ms/step - loss: 1.0874 - accuracy: 0.5033 - val_loss: 1.1124 - val_accuracy: 0.5079 - lr: 0.0100
Epoch 3/20
473/473 [==============================] - 74s 157ms/step - loss: 0.9372 - accuracy: 0.5949 - val_loss: 0.9057 - val_accuracy: 0.6277 - lr: 0.0100
Epoch 4/20
473/473 [==============================] - 72s 152ms/step - loss: 0.8693 - accuracy: 0.6280 - val_loss: 0.9304 - val_accuracy: 0.6130 - lr: 0.0100
Epoch 5/20
473/473 [==============================] - 69s 146ms/step - loss: 0.8381 - accuracy: 0.6477 - val_loss: 0.9043 - val_accuracy: 0.6299 - lr: 0.0100
Epoch 6/20
473/473 [==============================] - 68s 143ms/step - loss: 0.8052 - accuracy: 0.6638 - val_loss: 1.0157 - val_accuracy: 0.5843 - lr: 0.0100
Epoch 7/20
473/473 [==============================] - 67s 142ms/step - loss: 0.7746 - accuracy: 0.6803 - val_loss: 0.8872 - val_accuracy: 0.6472 - lr: 0.0100
Epoch 8/20
473/473 [==============================] - 67s 141ms/step - loss: 0.7455 - accuracy: 0.6926 - val_loss: 0.8119 - val_accuracy: 0.6723 - lr: 0.0100
Epoch 9/20
473/473 [==============================] - 67s 141ms/step - loss: 0.7258 - accuracy: 0.7029 - val_loss: 1.6141 - val_accuracy: 0.5612 - lr: 0.0100
Epoch 10/20
473/473 [==============================] - 71s 149ms/step - loss: 0.7120 - accuracy: 0.7115 - val_loss: 0.8237 - val_accuracy: 0.6558 - lr: 0.0100
Epoch 11/20
473/473 [==============================] - 69s 146ms/step - loss: 0.6929 - accuracy: 0.7186 - val_loss: 1.1831 - val_accuracy: 0.5001 - lr: 0.0100
Epoch 12/20
473/473 [==============================] - 68s 144ms/step - loss: 0.6094 - accuracy: 0.7522 - val_loss: 0.7039 - val_accuracy: 0.7085 - lr: 0.0020
Epoch 13/20
473/473 [==============================] - 68s 144ms/step - loss: 0.5724 - accuracy: 0.7721 - val_loss: 0.7011 - val_accuracy: 0.7237 - lr: 0.0020
Epoch 14/20
473/473 [==============================] - 71s 149ms/step - loss: 0.5578 - accuracy: 0.7740 - val_loss: 0.7767 - val_accuracy: 0.6958 - lr: 0.0020
Epoch 15/20
473/473 [==============================] - 71s 149ms/step - loss: 0.5400 - accuracy: 0.7831 - val_loss: 0.7318 - val_accuracy: 0.7267 - lr: 0.0020
Epoch 16/20
473/473 [==============================] - 68s 144ms/step - loss: 0.5205 - accuracy: 0.7915 - val_loss: 0.7206 - val_accuracy: 0.7314 - lr: 0.0020
Epoch 17/20
473/473 [==============================] - 69s 146ms/step - loss: 0.4907 - accuracy: 0.8020 - val_loss: 0.7196 - val_accuracy: 0.7304 - lr: 4.0000e-04
Epoch 18/20
473/473 [==============================] - 72s 152ms/step - loss: 0.4783 - accuracy: 0.8084 - val_loss: 0.7350 - val_accuracy: 0.7302 - lr: 4.0000e-04
1/1 [==============================] - 0s 65ms/step - loss: 0.6166 - accuracy: 0.8125
1/1 [==============================] - 0s 172ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 1 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 2 3 1 2 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 1.00 0.90 0.95 10
neutral 0.44 0.80 0.57 5
sad 0.88 0.70 0.78 10
surprise 1.00 0.86 0.92 7
accuracy 0.81 32
macro avg 0.83 0.81 0.80 32
weighted avg 0.87 0.81 0.83 32
batch_size = 64
model, results, history = model_building_and_evaluating_process(model_builder_enhanced_cnn, f'Model 2: Enhanced Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_enhanced_cnn.update(model)
metrics_enhanced_cnn = pd.concat([metrics_enhanced_cnn, results])
history_enhanced_cnn.update(history)
---------------------------------------------
Model 2: Enhanced Neural Network (64) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 3328
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 391,652
Trainable params: 390,692
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 71s 290ms/step - loss: 1.4343 - accuracy: 0.3738 - val_loss: 1.3023 - val_accuracy: 0.4097 - lr: 0.0100
Epoch 2/20
237/237 [==============================] - 68s 287ms/step - loss: 1.0529 - accuracy: 0.5253 - val_loss: 1.1257 - val_accuracy: 0.4959 - lr: 0.0100
Epoch 3/20
237/237 [==============================] - 66s 279ms/step - loss: 0.9480 - accuracy: 0.5859 - val_loss: 1.0692 - val_accuracy: 0.5373 - lr: 0.0100
Epoch 4/20
237/237 [==============================] - 68s 288ms/step - loss: 0.8785 - accuracy: 0.6289 - val_loss: 0.9133 - val_accuracy: 0.6080 - lr: 0.0100
Epoch 5/20
237/237 [==============================] - 66s 278ms/step - loss: 0.8345 - accuracy: 0.6466 - val_loss: 0.9580 - val_accuracy: 0.5672 - lr: 0.0100
Epoch 6/20
237/237 [==============================] - 66s 279ms/step - loss: 0.7965 - accuracy: 0.6662 - val_loss: 1.0283 - val_accuracy: 0.5393 - lr: 0.0100
Epoch 7/20
237/237 [==============================] - 66s 280ms/step - loss: 0.7560 - accuracy: 0.6842 - val_loss: 0.9096 - val_accuracy: 0.6034 - lr: 0.0100
Epoch 8/20
237/237 [==============================] - 67s 282ms/step - loss: 0.7458 - accuracy: 0.6911 - val_loss: 0.9428 - val_accuracy: 0.6223 - lr: 0.0100
Epoch 9/20
237/237 [==============================] - 67s 280ms/step - loss: 0.7233 - accuracy: 0.7022 - val_loss: 0.8124 - val_accuracy: 0.6665 - lr: 0.0100
Epoch 10/20
237/237 [==============================] - 65s 274ms/step - loss: 0.6904 - accuracy: 0.7161 - val_loss: 0.8657 - val_accuracy: 0.6345 - lr: 0.0100
Epoch 11/20
237/237 [==============================] - 64s 269ms/step - loss: 0.6786 - accuracy: 0.7200 - val_loss: 1.0323 - val_accuracy: 0.5746 - lr: 0.0100
Epoch 12/20
237/237 [==============================] - 64s 270ms/step - loss: 0.6735 - accuracy: 0.7235 - val_loss: 0.7623 - val_accuracy: 0.6914 - lr: 0.0100
Epoch 13/20
237/237 [==============================] - 64s 272ms/step - loss: 0.6468 - accuracy: 0.7343 - val_loss: 0.8515 - val_accuracy: 0.6506 - lr: 0.0100
Epoch 14/20
237/237 [==============================] - 66s 280ms/step - loss: 0.6523 - accuracy: 0.7341 - val_loss: 0.8055 - val_accuracy: 0.6735 - lr: 0.0100
Epoch 15/20
237/237 [==============================] - 67s 283ms/step - loss: 0.6228 - accuracy: 0.7449 - val_loss: 0.7623 - val_accuracy: 0.6934 - lr: 0.0100
Epoch 16/20
237/237 [==============================] - 66s 278ms/step - loss: 0.5357 - accuracy: 0.7836 - val_loss: 0.7076 - val_accuracy: 0.7179 - lr: 0.0020
Epoch 17/20
237/237 [==============================] - 67s 281ms/step - loss: 0.5034 - accuracy: 0.7944 - val_loss: 0.7176 - val_accuracy: 0.7195 - lr: 0.0020
Epoch 18/20
237/237 [==============================] - 68s 287ms/step - loss: 0.4814 - accuracy: 0.8036 - val_loss: 0.7248 - val_accuracy: 0.7197 - lr: 0.0020
Epoch 19/20
237/237 [==============================] - 65s 276ms/step - loss: 0.4638 - accuracy: 0.8125 - val_loss: 0.7596 - val_accuracy: 0.7169 - lr: 0.0020
Epoch 20/20
237/237 [==============================] - 66s 278ms/step - loss: 0.4301 - accuracy: 0.8266 - val_loss: 0.7374 - val_accuracy: 0.7368 - lr: 4.0000e-04
1/1 [==============================] - 0s 59ms/step - loss: 0.8335 - accuracy: 0.6875
1/1 [==============================] - 0s 168ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 0 0 0 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 2 2 1 1 0 2 1 0 0 1 0 2]
precision recall f1-score support
happy 0.75 0.90 0.82 10
neutral 0.33 0.40 0.36 5
sad 0.70 0.70 0.70 10
surprise 1.00 0.57 0.73 7
accuracy 0.69 32
macro avg 0.70 0.64 0.65 32
weighted avg 0.72 0.69 0.69 32
---------------------------------------------
Model 2: Enhanced Neural Network (64) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 1280
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 389,604
Trainable params: 388,644
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 67s 275ms/step - loss: 1.4771 - accuracy: 0.3600 - val_loss: 1.4708 - val_accuracy: 0.3221 - lr: 0.0100
Epoch 2/20
237/237 [==============================] - 66s 279ms/step - loss: 1.1109 - accuracy: 0.4953 - val_loss: 1.1091 - val_accuracy: 0.4850 - lr: 0.0100
Epoch 3/20
237/237 [==============================] - 67s 281ms/step - loss: 0.9491 - accuracy: 0.5890 - val_loss: 1.3789 - val_accuracy: 0.3982 - lr: 0.0100
Epoch 4/20
237/237 [==============================] - 67s 281ms/step - loss: 0.8670 - accuracy: 0.6362 - val_loss: 1.2985 - val_accuracy: 0.4947 - lr: 0.0100
Epoch 5/20
237/237 [==============================] - 66s 279ms/step - loss: 0.8030 - accuracy: 0.6671 - val_loss: 0.8474 - val_accuracy: 0.6480 - lr: 0.0100
Epoch 6/20
237/237 [==============================] - 65s 275ms/step - loss: 0.7823 - accuracy: 0.6764 - val_loss: 1.2658 - val_accuracy: 0.4623 - lr: 0.0100
Epoch 7/20
237/237 [==============================] - 67s 281ms/step - loss: 0.7453 - accuracy: 0.6904 - val_loss: 1.1134 - val_accuracy: 0.5017 - lr: 0.0100
Epoch 8/20
237/237 [==============================] - 67s 282ms/step - loss: 0.7310 - accuracy: 0.6989 - val_loss: 0.8642 - val_accuracy: 0.6430 - lr: 0.0100
Epoch 9/20
237/237 [==============================] - 67s 284ms/step - loss: 0.6453 - accuracy: 0.7391 - val_loss: 0.7260 - val_accuracy: 0.7060 - lr: 0.0020
Epoch 10/20
237/237 [==============================] - 65s 274ms/step - loss: 0.6114 - accuracy: 0.7509 - val_loss: 0.7420 - val_accuracy: 0.6990 - lr: 0.0020
Epoch 11/20
237/237 [==============================] - 65s 272ms/step - loss: 0.5994 - accuracy: 0.7577 - val_loss: 0.7393 - val_accuracy: 0.6996 - lr: 0.0020
Epoch 12/20
237/237 [==============================] - 66s 279ms/step - loss: 0.5818 - accuracy: 0.7635 - val_loss: 0.7339 - val_accuracy: 0.7071 - lr: 0.0020
Epoch 13/20
237/237 [==============================] - 66s 280ms/step - loss: 0.5456 - accuracy: 0.7798 - val_loss: 0.7099 - val_accuracy: 0.7219 - lr: 4.0000e-04
Epoch 14/20
237/237 [==============================] - 67s 284ms/step - loss: 0.5328 - accuracy: 0.7842 - val_loss: 0.7088 - val_accuracy: 0.7193 - lr: 4.0000e-04
Epoch 15/20
237/237 [==============================] - 69s 290ms/step - loss: 0.5242 - accuracy: 0.7880 - val_loss: 0.7054 - val_accuracy: 0.7277 - lr: 4.0000e-04
Epoch 16/20
237/237 [==============================] - 66s 278ms/step - loss: 0.5188 - accuracy: 0.7888 - val_loss: 0.7322 - val_accuracy: 0.7177 - lr: 4.0000e-04
Epoch 17/20
237/237 [==============================] - 66s 280ms/step - loss: 0.5141 - accuracy: 0.7944 - val_loss: 0.7164 - val_accuracy: 0.7167 - lr: 4.0000e-04
Epoch 18/20
237/237 [==============================] - 65s 273ms/step - loss: 0.5106 - accuracy: 0.7966 - val_loss: 0.7246 - val_accuracy: 0.7227 - lr: 4.0000e-04
Epoch 19/20
237/237 [==============================] - 65s 273ms/step - loss: 0.4973 - accuracy: 0.8042 - val_loss: 0.7274 - val_accuracy: 0.7221 - lr: 8.0000e-05
Epoch 20/20
237/237 [==============================] - 65s 276ms/step - loss: 0.4975 - accuracy: 0.7991 - val_loss: 0.7241 - val_accuracy: 0.7219 - lr: 8.0000e-05
1/1 [==============================] - 0s 59ms/step - loss: 0.5514 - accuracy: 0.7500
1/1 [==============================] - 0s 166ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 2 3 1 2 0 1 1 0 0 1 0 2]
precision recall f1-score support
happy 0.90 0.90 0.90 10
neutral 0.29 0.40 0.33 5
sad 0.78 0.70 0.74 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.74 0.71 0.72 32
weighted avg 0.79 0.75 0.77 32
batch_size = 128
model, results, history = model_building_and_evaluating_process(model_builder_enhanced_cnn, f'Model 2: Enhanced Neural Network ({batch_size})',
batch_size=batch_size, epochs=20)
model_enhanced_cnn.update(model)
metrics_enhanced_cnn = pd.concat([metrics_enhanced_cnn, results])
history_enhanced_cnn.update(history)
---------------------------------------------
Model 2: Enhanced Neural Network (128) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 3328
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 391,652
Trainable params: 390,692
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 64s 519ms/step - loss: 1.6542 - accuracy: 0.3290 - val_loss: 2.4356 - val_accuracy: 0.2767 - lr: 0.0100
Epoch 2/20
119/119 [==============================] - 61s 512ms/step - loss: 1.1971 - accuracy: 0.4397 - val_loss: 1.4953 - val_accuracy: 0.3705 - lr: 0.0100
Epoch 3/20
119/119 [==============================] - 61s 515ms/step - loss: 0.9932 - accuracy: 0.5528 - val_loss: 1.2061 - val_accuracy: 0.4422 - lr: 0.0100
Epoch 4/20
119/119 [==============================] - 64s 536ms/step - loss: 0.9217 - accuracy: 0.6010 - val_loss: 1.1810 - val_accuracy: 0.4862 - lr: 0.0100
Epoch 5/20
119/119 [==============================] - 63s 527ms/step - loss: 0.8618 - accuracy: 0.6308 - val_loss: 1.0730 - val_accuracy: 0.5421 - lr: 0.0100
Epoch 6/20
119/119 [==============================] - 62s 522ms/step - loss: 0.8079 - accuracy: 0.6564 - val_loss: 1.1524 - val_accuracy: 0.5228 - lr: 0.0100
Epoch 7/20
119/119 [==============================] - 61s 511ms/step - loss: 0.7682 - accuracy: 0.6809 - val_loss: 0.8772 - val_accuracy: 0.6488 - lr: 0.0100
Epoch 8/20
119/119 [==============================] - 62s 519ms/step - loss: 0.7431 - accuracy: 0.6910 - val_loss: 0.8538 - val_accuracy: 0.6484 - lr: 0.0100
Epoch 9/20
119/119 [==============================] - 62s 519ms/step - loss: 0.7175 - accuracy: 0.7035 - val_loss: 0.9057 - val_accuracy: 0.6367 - lr: 0.0100
Epoch 10/20
119/119 [==============================] - 61s 515ms/step - loss: 0.6923 - accuracy: 0.7154 - val_loss: 0.8409 - val_accuracy: 0.6381 - lr: 0.0100
Epoch 11/20
119/119 [==============================] - 61s 508ms/step - loss: 0.6918 - accuracy: 0.7132 - val_loss: 1.1288 - val_accuracy: 0.5558 - lr: 0.0100
Epoch 12/20
119/119 [==============================] - 61s 515ms/step - loss: 0.6682 - accuracy: 0.7259 - val_loss: 0.7400 - val_accuracy: 0.7083 - lr: 0.0100
Epoch 13/20
119/119 [==============================] - 62s 518ms/step - loss: 0.6393 - accuracy: 0.7362 - val_loss: 1.1642 - val_accuracy: 0.5188 - lr: 0.0100
Epoch 14/20
119/119 [==============================] - 62s 522ms/step - loss: 0.6471 - accuracy: 0.7364 - val_loss: 0.8443 - val_accuracy: 0.6552 - lr: 0.0100
Epoch 15/20
119/119 [==============================] - 63s 528ms/step - loss: 0.6339 - accuracy: 0.7405 - val_loss: 1.0644 - val_accuracy: 0.5449 - lr: 0.0100
Epoch 16/20
119/119 [==============================] - 65s 548ms/step - loss: 0.5522 - accuracy: 0.7772 - val_loss: 0.7175 - val_accuracy: 0.7185 - lr: 0.0020
Epoch 17/20
119/119 [==============================] - 63s 533ms/step - loss: 0.5399 - accuracy: 0.7806 - val_loss: 0.7433 - val_accuracy: 0.7203 - lr: 0.0020
Epoch 18/20
119/119 [==============================] - 63s 525ms/step - loss: 0.5165 - accuracy: 0.7950 - val_loss: 0.7341 - val_accuracy: 0.7171 - lr: 0.0020
Epoch 19/20
119/119 [==============================] - 63s 530ms/step - loss: 0.4966 - accuracy: 0.7983 - val_loss: 0.7138 - val_accuracy: 0.7292 - lr: 0.0020
Epoch 20/20
119/119 [==============================] - 63s 529ms/step - loss: 0.4891 - accuracy: 0.8049 - val_loss: 0.8095 - val_accuracy: 0.7024 - lr: 0.0020
1/1 [==============================] - 0s 58ms/step - loss: 0.7810 - accuracy: 0.7812
1/1 [==============================] - 0s 166ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 2 3 0 1 3 2 1 0 1 0 3 2 2 3 0 3 2 0 2 2 2 0 2 0 0 1 0 0 2 0 2]
precision recall f1-score support
happy 0.83 1.00 0.91 10
neutral 0.50 0.40 0.44 5
sad 0.73 0.80 0.76 10
surprise 1.00 0.71 0.83 7
accuracy 0.78 32
macro avg 0.77 0.73 0.74 32
weighted avg 0.78 0.78 0.77 32
---------------------------------------------
Model 2: Enhanced Neural Network (128) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 256) 1280
batch_normalization (BatchN (None, 48, 48, 256) 1024
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 256) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 256) 0
)
conv2d_1 (Conv2D) (None, 24, 24, 128) 131200
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
conv2d_2 (Conv2D) (None, 12, 12, 64) 32832
batch_normalization_2 (Batc (None, 12, 12, 64) 256
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 64) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 64) 0
2D)
conv2d_3 (Conv2D) (None, 6, 6, 32) 8224
batch_normalization_3 (Batc (None, 6, 6, 32) 128
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 32) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 32) 0
2D)
flatten (Flatten) (None, 288) 0
dense (Dense) (None, 512) 147968
dense_1 (Dense) (None, 128) 65664
dense_2 (Dense) (None, 4) 516
=================================================================
Total params: 389,604
Trainable params: 388,644
Non-trainable params: 960
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 66s 538ms/step - loss: 1.6232 - accuracy: 0.3075 - val_loss: 1.3630 - val_accuracy: 0.3719 - lr: 0.0100
Epoch 2/20
119/119 [==============================] - 63s 531ms/step - loss: 1.3055 - accuracy: 0.3508 - val_loss: 1.3287 - val_accuracy: 0.3661 - lr: 0.0100
Epoch 3/20
119/119 [==============================] - 63s 528ms/step - loss: 1.1279 - accuracy: 0.4720 - val_loss: 1.7102 - val_accuracy: 0.2885 - lr: 0.0100
Epoch 4/20
119/119 [==============================] - 63s 530ms/step - loss: 0.9815 - accuracy: 0.5736 - val_loss: 1.1245 - val_accuracy: 0.4917 - lr: 0.0100
Epoch 5/20
119/119 [==============================] - 64s 535ms/step - loss: 0.8900 - accuracy: 0.6237 - val_loss: 1.3265 - val_accuracy: 0.4374 - lr: 0.0100
Epoch 6/20
119/119 [==============================] - 63s 527ms/step - loss: 0.8142 - accuracy: 0.6593 - val_loss: 0.9534 - val_accuracy: 0.5994 - lr: 0.0100
Epoch 7/20
119/119 [==============================] - 62s 524ms/step - loss: 0.7780 - accuracy: 0.6763 - val_loss: 0.8655 - val_accuracy: 0.6317 - lr: 0.0100
Epoch 8/20
119/119 [==============================] - 62s 524ms/step - loss: 0.7418 - accuracy: 0.6944 - val_loss: 0.9340 - val_accuracy: 0.6219 - lr: 0.0100
Epoch 9/20
119/119 [==============================] - 62s 521ms/step - loss: 0.7574 - accuracy: 0.6847 - val_loss: 0.9271 - val_accuracy: 0.6259 - lr: 0.0100
Epoch 10/20
119/119 [==============================] - 62s 522ms/step - loss: 0.7046 - accuracy: 0.7081 - val_loss: 1.6188 - val_accuracy: 0.4278 - lr: 0.0100
Epoch 11/20
119/119 [==============================] - 63s 527ms/step - loss: 0.6439 - accuracy: 0.7373 - val_loss: 0.7312 - val_accuracy: 0.7091 - lr: 0.0020
Epoch 12/20
119/119 [==============================] - 63s 530ms/step - loss: 0.6138 - accuracy: 0.7464 - val_loss: 0.7014 - val_accuracy: 0.7165 - lr: 0.0020
Epoch 13/20
119/119 [==============================] - 63s 532ms/step - loss: 0.5905 - accuracy: 0.7578 - val_loss: 0.7234 - val_accuracy: 0.7133 - lr: 0.0020
Epoch 14/20
119/119 [==============================] - 63s 529ms/step - loss: 0.5896 - accuracy: 0.7609 - val_loss: 0.7147 - val_accuracy: 0.7187 - lr: 0.0020
Epoch 15/20
119/119 [==============================] - 62s 524ms/step - loss: 0.5730 - accuracy: 0.7642 - val_loss: 0.7030 - val_accuracy: 0.7227 - lr: 0.0020
Epoch 16/20
119/119 [==============================] - 62s 524ms/step - loss: 0.5514 - accuracy: 0.7771 - val_loss: 0.7115 - val_accuracy: 0.7217 - lr: 4.0000e-04
Epoch 17/20
119/119 [==============================] - 62s 525ms/step - loss: 0.5377 - accuracy: 0.7799 - val_loss: 0.7362 - val_accuracy: 0.7153 - lr: 4.0000e-04
1/1 [==============================] - 0s 60ms/step - loss: 0.7239 - accuracy: 0.7500
1/1 [==============================] - 0s 162ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 0 3 2 1 0 1 0 3 2 0 3 0 3 2 0 2 2 3 1 1 0 0 1 0 1 0 0 2]
precision recall f1-score support
happy 0.77 1.00 0.87 10
neutral 0.43 0.60 0.50 5
sad 0.83 0.50 0.62 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.76 0.74 0.73 32
weighted avg 0.79 0.75 0.75 32
model_name = 'Model 2: Enhanced Neural Network'
save_object(history_enhanced_cnn, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metrics_enhanced_cnn, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_enhanced_cnn, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_enhanced_cnn, data_type='Training Set')
loss_and_accuracy_comparisson(history_enhanced_cnn, data_type='Validation Set')
metrics_enhanced_cnn.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 2: Enhanced Neural Network (32) - RGB | 0.792309 | 0.727145 | 0.843750 |
| Model 2: Enhanced Neural Network (32) - GRAY | 0.791515 | 0.731364 | 0.812500 |
| Model 2: Enhanced Neural Network (64) - RGB | 0.826593 | 0.736789 | 0.687500 |
| Model 2: Enhanced Neural Network (64) - GRAY | 0.788007 | 0.727748 | 0.750000 |
| Model 2: Enhanced Neural Network (128) - RGB | 0.798332 | 0.729154 | 0.781250 |
| Model 2: Enhanced Neural Network (128) - GRAY | 0.764180 | 0.722725 | 0.750000 |
⏩ Observations and Insights:
⏩ Recommendations:
Think About It:
⏩ Observations and Insights:
In this section, we will create several Transfer Learning architectures. For the pre-trained models, we will select three popular architectures namely, VGG16, ResNet v2, and Efficient Net. The difference between these architectures and the previous architectures is that these will require 3 input channels while the earlier ones worked on 'grayscale' images. Therefore, we need to create new DataLoaders.
In this section, we are creating data loaders that we will use as inputs to our Neural Network. We will have to go with color_mode = 'rgb' as this is the required format for the transfer learning architectures.
train_generator_rgb, _, _ = create_data_loaders(IMG_DIR, 'rgb')
# Taking a look at some examples of our augmented training data with color_mode='rgb'.
images, labels = next(train_generator_rgb)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image)
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=rgb: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
VGG16(weights='imagenet', include_top=False, input_shape = (48, 48, 3)).summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
58889256/58889256 [==============================] - 0s 0us/step
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 48, 48, 3)] 0
block1_conv1 (Conv2D) (None, 48, 48, 64) 1792
block1_conv2 (Conv2D) (None, 48, 48, 64) 36928
block1_pool (MaxPooling2D) (None, 24, 24, 64) 0
block2_conv1 (Conv2D) (None, 24, 24, 128) 73856
block2_conv2 (Conv2D) (None, 24, 24, 128) 147584
block2_pool (MaxPooling2D) (None, 12, 12, 128) 0
block3_conv1 (Conv2D) (None, 12, 12, 256) 295168
block3_conv2 (Conv2D) (None, 12, 12, 256) 590080
block3_conv3 (Conv2D) (None, 12, 12, 256) 590080
block3_pool (MaxPooling2D) (None, 6, 6, 256) 0
block4_conv1 (Conv2D) (None, 6, 6, 512) 1180160
block4_conv2 (Conv2D) (None, 6, 6, 512) 2359808
block4_conv3 (Conv2D) (None, 6, 6, 512) 2359808
block4_pool (MaxPooling2D) (None, 3, 3, 512) 0
block5_conv1 (Conv2D) (None, 3, 3, 512) 2359808
block5_conv2 (Conv2D) (None, 3, 3, 512) 2359808
block5_conv3 (Conv2D) (None, 3, 3, 512) 2359808
block5_pool (MaxPooling2D) (None, 1, 1, 512) 0
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________
# Model 3: VGG16
def model_builder_vgg16(input_shape, tune=0):
"""Transfer approach with the VGG16 Model
"""
vgg_model = VGG16(weights='imagenet', include_top=False, input_shape = input_shape)
# Making all the layers of the VGG model non-trainable
if tune > 0:
for layer in vgg_model.layers[:-tune]:
layer.trainable = False
else:
for layer in vgg_model.layers:
layer.trainable = False
model = Sequential([
vgg_model,
Flatten(),
Dense(256, activation='relu'),
Dense(128, activation='relu'),
Dropout(0.3),
Dense(64, activation='relu'),
BatchNormalization(),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_vgg16, metric_vgg16, history_vgg16 = model_building_and_evaluating_process(
model_builder_vgg16, f'Model 3: VGG16 ({batch_size})', batch_size=batch_size, epochs=20, include_grayscale=False
)
# model_vgg16['Model 3: VGG16 (32) - RGB'].get_metrics_result()['accuracy'].numpy()
---------------------------------------------
Model 3: VGG16 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 1, 1, 512) 14714688
flatten (Flatten) (None, 512) 0
dense (Dense) (None, 256) 131328
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 14,887,684
Trainable params: 172,868
Non-trainable params: 14,714,816
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 41s 83ms/step - loss: 1.2524 - accuracy: 0.4357 - val_loss: 1.0932 - val_accuracy: 0.5252 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 38s 80ms/step - loss: 1.1323 - accuracy: 0.5042 - val_loss: 1.2063 - val_accuracy: 0.4663 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 37s 79ms/step - loss: 1.0921 - accuracy: 0.5343 - val_loss: 1.1006 - val_accuracy: 0.5186 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 39s 82ms/step - loss: 1.0750 - accuracy: 0.5391 - val_loss: 1.0562 - val_accuracy: 0.5431 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 38s 80ms/step - loss: 1.0516 - accuracy: 0.5501 - val_loss: 1.0475 - val_accuracy: 0.5479 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 38s 80ms/step - loss: 1.0294 - accuracy: 0.5634 - val_loss: 1.0337 - val_accuracy: 0.5588 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 37s 79ms/step - loss: 1.0172 - accuracy: 0.5627 - val_loss: 1.1040 - val_accuracy: 0.5236 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 37s 78ms/step - loss: 1.0022 - accuracy: 0.5729 - val_loss: 1.0965 - val_accuracy: 0.5250 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 37s 78ms/step - loss: 0.9757 - accuracy: 0.5891 - val_loss: 1.0464 - val_accuracy: 0.5437 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 38s 80ms/step - loss: 0.9240 - accuracy: 0.6129 - val_loss: 1.0171 - val_accuracy: 0.5668 - lr: 2.0000e-04
Epoch 11/20
473/473 [==============================] - 37s 79ms/step - loss: 0.8960 - accuracy: 0.6282 - val_loss: 1.0418 - val_accuracy: 0.5531 - lr: 2.0000e-04
Epoch 12/20
473/473 [==============================] - 37s 78ms/step - loss: 0.8756 - accuracy: 0.6335 - val_loss: 1.0299 - val_accuracy: 0.5576 - lr: 2.0000e-04
Epoch 13/20
473/473 [==============================] - 37s 79ms/step - loss: 0.8700 - accuracy: 0.6419 - val_loss: 1.0422 - val_accuracy: 0.5622 - lr: 2.0000e-04
Epoch 14/20
473/473 [==============================] - 38s 80ms/step - loss: 0.8420 - accuracy: 0.6513 - val_loss: 1.0207 - val_accuracy: 0.5732 - lr: 4.0000e-05
Epoch 15/20
473/473 [==============================] - 37s 78ms/step - loss: 0.8319 - accuracy: 0.6550 - val_loss: 1.0324 - val_accuracy: 0.5644 - lr: 4.0000e-05
1/1 [==============================] - 0s 71ms/step - loss: 0.8450 - accuracy: 0.6250
1/1 [==============================] - 0s 225ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 2 0 1 2 3 1 1 0 2 1 3 2 2 3 0 3 2 0 2 1 3 1 2 1 2 1 0 2 1 0 2]
precision recall f1-score support
happy 0.86 0.60 0.71 10
neutral 0.22 0.40 0.29 5
sad 0.64 0.70 0.67 10
surprise 1.00 0.71 0.83 7
accuracy 0.62 32
macro avg 0.68 0.60 0.62 32
weighted avg 0.72 0.62 0.66 32
# Let's try again tuning some of the last layers in the transfer model
batch_size = 32
model, results, history = model_building_and_evaluating_process(model_builder_vgg16, f'Model 3: Tuned VGG16 ({batch_size})',
batch_size=batch_size, epochs=20, tune=4, include_grayscale=False)
model_vgg16.update(model)
metric_vgg16 = pd.concat([metric_vgg16, results])
history_vgg16.update(history)
---------------------------------------------
Model 3: Tuned VGG16 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 1, 1, 512) 14714688
flatten (Flatten) (None, 512) 0
dense (Dense) (None, 256) 131328
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 14,887,684
Trainable params: 7,252,292
Non-trainable params: 7,635,392
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 60s 123ms/step - loss: 1.1853 - accuracy: 0.4585 - val_loss: 0.9170 - val_accuracy: 0.5789 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 58s 122ms/step - loss: 1.0514 - accuracy: 0.5244 - val_loss: 0.9705 - val_accuracy: 0.5971 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 58s 122ms/step - loss: 0.9214 - accuracy: 0.5940 - val_loss: 0.8417 - val_accuracy: 0.6438 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 56s 118ms/step - loss: 0.8536 - accuracy: 0.6485 - val_loss: 0.9996 - val_accuracy: 0.5863 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 57s 121ms/step - loss: 0.8037 - accuracy: 0.6740 - val_loss: 0.8071 - val_accuracy: 0.6713 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 57s 121ms/step - loss: 0.7712 - accuracy: 0.6879 - val_loss: 0.7755 - val_accuracy: 0.6769 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 58s 122ms/step - loss: 0.7429 - accuracy: 0.7063 - val_loss: 0.7747 - val_accuracy: 0.6799 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 56s 119ms/step - loss: 0.7550 - accuracy: 0.6960 - val_loss: 0.9831 - val_accuracy: 0.5990 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 56s 118ms/step - loss: 0.7480 - accuracy: 0.7009 - val_loss: 1.0221 - val_accuracy: 0.5720 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 57s 120ms/step - loss: 0.7018 - accuracy: 0.7259 - val_loss: 0.7804 - val_accuracy: 0.6888 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 57s 121ms/step - loss: 0.6038 - accuracy: 0.7641 - val_loss: 0.7710 - val_accuracy: 0.7022 - lr: 2.0000e-04
Epoch 12/20
473/473 [==============================] - 58s 122ms/step - loss: 0.5686 - accuracy: 0.7844 - val_loss: 0.7590 - val_accuracy: 0.7129 - lr: 2.0000e-04
Epoch 13/20
473/473 [==============================] - 56s 119ms/step - loss: 0.5485 - accuracy: 0.7941 - val_loss: 0.7905 - val_accuracy: 0.6954 - lr: 2.0000e-04
Epoch 14/20
473/473 [==============================] - 56s 119ms/step - loss: 0.5202 - accuracy: 0.8088 - val_loss: 0.8029 - val_accuracy: 0.7022 - lr: 2.0000e-04
Epoch 15/20
473/473 [==============================] - 56s 118ms/step - loss: 0.5028 - accuracy: 0.8125 - val_loss: 0.8014 - val_accuracy: 0.7026 - lr: 2.0000e-04
Epoch 16/20
473/473 [==============================] - 56s 119ms/step - loss: 0.4603 - accuracy: 0.8345 - val_loss: 0.8063 - val_accuracy: 0.7016 - lr: 4.0000e-05
Epoch 17/20
473/473 [==============================] - 56s 119ms/step - loss: 0.4422 - accuracy: 0.8435 - val_loss: 0.8098 - val_accuracy: 0.7048 - lr: 4.0000e-05
1/1 [==============================] - 0s 78ms/step - loss: 0.5211 - accuracy: 0.8125
1/1 [==============================] - 0s 240ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 1 3 1 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 0 1 0 1 1 0 1 1 0 2]
precision recall f1-score support
happy 0.90 0.90 0.90 10
neutral 0.50 1.00 0.67 5
sad 1.00 0.60 0.75 10
surprise 1.00 0.86 0.92 7
accuracy 0.81 32
macro avg 0.85 0.84 0.81 32
weighted avg 0.89 0.81 0.82 32
model_name = 'Model 3: VGG16'
save_object(history_vgg16, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_vgg16, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_vgg16, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_vgg16, data_type='Training Set')
loss_and_accuracy_comparisson(history_vgg16, data_type='Validation Set')
metric_vgg16.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 3: VGG16 (32) - RGB | 0.651267 | 0.573237 | 0.625000 |
| Model 3: Tuned VGG16 (32) - RGB | 0.784367 | 0.712879 | 0.812500 |
Think About It:
Note: You can even go back and build your own architecture on top of the VGG16 Transfer layer and see if you can improve the performance
⏩ Observations and Insights:
⏩ Recommendations:
ap.ResNet101(include_top=False, weights="imagenet", input_shape=(48, 48, 3)).summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/resnet/resnet101_weights_tf_dim_ordering_tf_kernels_notop.h5
171446536/171446536 [==============================] - 1s 0us/step
Model: "resnet101"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 48, 48, 3)] 0 []
conv1_pad (ZeroPadding2D) (None, 54, 54, 3) 0 ['input_2[0][0]']
conv1_conv (Conv2D) (None, 24, 24, 64) 9472 ['conv1_pad[0][0]']
conv1_bn (BatchNormalization) (None, 24, 24, 64) 256 ['conv1_conv[0][0]']
conv1_relu (Activation) (None, 24, 24, 64) 0 ['conv1_bn[0][0]']
pool1_pad (ZeroPadding2D) (None, 26, 26, 64) 0 ['conv1_relu[0][0]']
pool1_pool (MaxPooling2D) (None, 12, 12, 64) 0 ['pool1_pad[0][0]']
conv2_block1_1_conv (Conv2D) (None, 12, 12, 64) 4160 ['pool1_pool[0][0]']
conv2_block1_1_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block1_1_conv[0][0]']
ization)
conv2_block1_1_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block1_1_bn[0][0]']
n)
conv2_block1_2_conv (Conv2D) (None, 12, 12, 64) 36928 ['conv2_block1_1_relu[0][0]']
conv2_block1_2_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block1_2_conv[0][0]']
ization)
conv2_block1_2_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block1_2_bn[0][0]']
n)
conv2_block1_0_conv (Conv2D) (None, 12, 12, 256) 16640 ['pool1_pool[0][0]']
conv2_block1_3_conv (Conv2D) (None, 12, 12, 256) 16640 ['conv2_block1_2_relu[0][0]']
conv2_block1_0_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block1_0_conv[0][0]']
ization)
conv2_block1_3_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block1_3_conv[0][0]']
ization)
conv2_block1_add (Add) (None, 12, 12, 256) 0 ['conv2_block1_0_bn[0][0]',
'conv2_block1_3_bn[0][0]']
conv2_block1_out (Activation) (None, 12, 12, 256) 0 ['conv2_block1_add[0][0]']
conv2_block2_1_conv (Conv2D) (None, 12, 12, 64) 16448 ['conv2_block1_out[0][0]']
conv2_block2_1_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block2_1_conv[0][0]']
ization)
conv2_block2_1_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block2_1_bn[0][0]']
n)
conv2_block2_2_conv (Conv2D) (None, 12, 12, 64) 36928 ['conv2_block2_1_relu[0][0]']
conv2_block2_2_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block2_2_conv[0][0]']
ization)
conv2_block2_2_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block2_2_bn[0][0]']
n)
conv2_block2_3_conv (Conv2D) (None, 12, 12, 256) 16640 ['conv2_block2_2_relu[0][0]']
conv2_block2_3_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block2_3_conv[0][0]']
ization)
conv2_block2_add (Add) (None, 12, 12, 256) 0 ['conv2_block1_out[0][0]',
'conv2_block2_3_bn[0][0]']
conv2_block2_out (Activation) (None, 12, 12, 256) 0 ['conv2_block2_add[0][0]']
conv2_block3_1_conv (Conv2D) (None, 12, 12, 64) 16448 ['conv2_block2_out[0][0]']
conv2_block3_1_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block3_1_conv[0][0]']
ization)
conv2_block3_1_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block3_1_bn[0][0]']
n)
conv2_block3_2_conv (Conv2D) (None, 12, 12, 64) 36928 ['conv2_block3_1_relu[0][0]']
conv2_block3_2_bn (BatchNormal (None, 12, 12, 64) 256 ['conv2_block3_2_conv[0][0]']
ization)
conv2_block3_2_relu (Activatio (None, 12, 12, 64) 0 ['conv2_block3_2_bn[0][0]']
n)
conv2_block3_3_conv (Conv2D) (None, 12, 12, 256) 16640 ['conv2_block3_2_relu[0][0]']
conv2_block3_3_bn (BatchNormal (None, 12, 12, 256) 1024 ['conv2_block3_3_conv[0][0]']
ization)
conv2_block3_add (Add) (None, 12, 12, 256) 0 ['conv2_block2_out[0][0]',
'conv2_block3_3_bn[0][0]']
conv2_block3_out (Activation) (None, 12, 12, 256) 0 ['conv2_block3_add[0][0]']
conv3_block1_1_conv (Conv2D) (None, 6, 6, 128) 32896 ['conv2_block3_out[0][0]']
conv3_block1_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block1_1_conv[0][0]']
ization)
conv3_block1_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block1_1_bn[0][0]']
n)
conv3_block1_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block1_1_relu[0][0]']
conv3_block1_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block1_2_conv[0][0]']
ization)
conv3_block1_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block1_2_bn[0][0]']
n)
conv3_block1_0_conv (Conv2D) (None, 6, 6, 512) 131584 ['conv2_block3_out[0][0]']
conv3_block1_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block1_2_relu[0][0]']
conv3_block1_0_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block1_0_conv[0][0]']
ization)
conv3_block1_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block1_3_conv[0][0]']
ization)
conv3_block1_add (Add) (None, 6, 6, 512) 0 ['conv3_block1_0_bn[0][0]',
'conv3_block1_3_bn[0][0]']
conv3_block1_out (Activation) (None, 6, 6, 512) 0 ['conv3_block1_add[0][0]']
conv3_block2_1_conv (Conv2D) (None, 6, 6, 128) 65664 ['conv3_block1_out[0][0]']
conv3_block2_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block2_1_conv[0][0]']
ization)
conv3_block2_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block2_1_bn[0][0]']
n)
conv3_block2_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block2_1_relu[0][0]']
conv3_block2_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block2_2_conv[0][0]']
ization)
conv3_block2_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block2_2_bn[0][0]']
n)
conv3_block2_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block2_2_relu[0][0]']
conv3_block2_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block2_3_conv[0][0]']
ization)
conv3_block2_add (Add) (None, 6, 6, 512) 0 ['conv3_block1_out[0][0]',
'conv3_block2_3_bn[0][0]']
conv3_block2_out (Activation) (None, 6, 6, 512) 0 ['conv3_block2_add[0][0]']
conv3_block3_1_conv (Conv2D) (None, 6, 6, 128) 65664 ['conv3_block2_out[0][0]']
conv3_block3_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block3_1_conv[0][0]']
ization)
conv3_block3_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block3_1_bn[0][0]']
n)
conv3_block3_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block3_1_relu[0][0]']
conv3_block3_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block3_2_conv[0][0]']
ization)
conv3_block3_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block3_2_bn[0][0]']
n)
conv3_block3_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block3_2_relu[0][0]']
conv3_block3_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block3_3_conv[0][0]']
ization)
conv3_block3_add (Add) (None, 6, 6, 512) 0 ['conv3_block2_out[0][0]',
'conv3_block3_3_bn[0][0]']
conv3_block3_out (Activation) (None, 6, 6, 512) 0 ['conv3_block3_add[0][0]']
conv3_block4_1_conv (Conv2D) (None, 6, 6, 128) 65664 ['conv3_block3_out[0][0]']
conv3_block4_1_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block4_1_conv[0][0]']
ization)
conv3_block4_1_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block4_1_bn[0][0]']
n)
conv3_block4_2_conv (Conv2D) (None, 6, 6, 128) 147584 ['conv3_block4_1_relu[0][0]']
conv3_block4_2_bn (BatchNormal (None, 6, 6, 128) 512 ['conv3_block4_2_conv[0][0]']
ization)
conv3_block4_2_relu (Activatio (None, 6, 6, 128) 0 ['conv3_block4_2_bn[0][0]']
n)
conv3_block4_3_conv (Conv2D) (None, 6, 6, 512) 66048 ['conv3_block4_2_relu[0][0]']
conv3_block4_3_bn (BatchNormal (None, 6, 6, 512) 2048 ['conv3_block4_3_conv[0][0]']
ization)
conv3_block4_add (Add) (None, 6, 6, 512) 0 ['conv3_block3_out[0][0]',
'conv3_block4_3_bn[0][0]']
conv3_block4_out (Activation) (None, 6, 6, 512) 0 ['conv3_block4_add[0][0]']
conv4_block1_1_conv (Conv2D) (None, 3, 3, 256) 131328 ['conv3_block4_out[0][0]']
conv4_block1_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block1_1_conv[0][0]']
ization)
conv4_block1_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block1_1_bn[0][0]']
n)
conv4_block1_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block1_1_relu[0][0]']
conv4_block1_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block1_2_conv[0][0]']
ization)
conv4_block1_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block1_2_bn[0][0]']
n)
conv4_block1_0_conv (Conv2D) (None, 3, 3, 1024) 525312 ['conv3_block4_out[0][0]']
conv4_block1_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block1_2_relu[0][0]']
conv4_block1_0_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block1_0_conv[0][0]']
ization)
conv4_block1_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block1_3_conv[0][0]']
ization)
conv4_block1_add (Add) (None, 3, 3, 1024) 0 ['conv4_block1_0_bn[0][0]',
'conv4_block1_3_bn[0][0]']
conv4_block1_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block1_add[0][0]']
conv4_block2_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block1_out[0][0]']
conv4_block2_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block2_1_conv[0][0]']
ization)
conv4_block2_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block2_1_bn[0][0]']
n)
conv4_block2_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block2_1_relu[0][0]']
conv4_block2_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block2_2_conv[0][0]']
ization)
conv4_block2_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block2_2_bn[0][0]']
n)
conv4_block2_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block2_2_relu[0][0]']
conv4_block2_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block2_3_conv[0][0]']
ization)
conv4_block2_add (Add) (None, 3, 3, 1024) 0 ['conv4_block1_out[0][0]',
'conv4_block2_3_bn[0][0]']
conv4_block2_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block2_add[0][0]']
conv4_block3_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block2_out[0][0]']
conv4_block3_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block3_1_conv[0][0]']
ization)
conv4_block3_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block3_1_bn[0][0]']
n)
conv4_block3_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block3_1_relu[0][0]']
conv4_block3_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block3_2_conv[0][0]']
ization)
conv4_block3_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block3_2_bn[0][0]']
n)
conv4_block3_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block3_2_relu[0][0]']
conv4_block3_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block3_3_conv[0][0]']
ization)
conv4_block3_add (Add) (None, 3, 3, 1024) 0 ['conv4_block2_out[0][0]',
'conv4_block3_3_bn[0][0]']
conv4_block3_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block3_add[0][0]']
conv4_block4_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block3_out[0][0]']
conv4_block4_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block4_1_conv[0][0]']
ization)
conv4_block4_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block4_1_bn[0][0]']
n)
conv4_block4_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block4_1_relu[0][0]']
conv4_block4_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block4_2_conv[0][0]']
ization)
conv4_block4_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block4_2_bn[0][0]']
n)
conv4_block4_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block4_2_relu[0][0]']
conv4_block4_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block4_3_conv[0][0]']
ization)
conv4_block4_add (Add) (None, 3, 3, 1024) 0 ['conv4_block3_out[0][0]',
'conv4_block4_3_bn[0][0]']
conv4_block4_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block4_add[0][0]']
conv4_block5_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block4_out[0][0]']
conv4_block5_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block5_1_conv[0][0]']
ization)
conv4_block5_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block5_1_bn[0][0]']
n)
conv4_block5_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block5_1_relu[0][0]']
conv4_block5_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block5_2_conv[0][0]']
ization)
conv4_block5_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block5_2_bn[0][0]']
n)
conv4_block5_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block5_2_relu[0][0]']
conv4_block5_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block5_3_conv[0][0]']
ization)
conv4_block5_add (Add) (None, 3, 3, 1024) 0 ['conv4_block4_out[0][0]',
'conv4_block5_3_bn[0][0]']
conv4_block5_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block5_add[0][0]']
conv4_block6_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block5_out[0][0]']
conv4_block6_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block6_1_conv[0][0]']
ization)
conv4_block6_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block6_1_bn[0][0]']
n)
conv4_block6_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block6_1_relu[0][0]']
conv4_block6_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block6_2_conv[0][0]']
ization)
conv4_block6_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block6_2_bn[0][0]']
n)
conv4_block6_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block6_2_relu[0][0]']
conv4_block6_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block6_3_conv[0][0]']
ization)
conv4_block6_add (Add) (None, 3, 3, 1024) 0 ['conv4_block5_out[0][0]',
'conv4_block6_3_bn[0][0]']
conv4_block6_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block6_add[0][0]']
conv4_block7_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block6_out[0][0]']
conv4_block7_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block7_1_conv[0][0]']
ization)
conv4_block7_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block7_1_bn[0][0]']
n)
conv4_block7_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block7_1_relu[0][0]']
conv4_block7_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block7_2_conv[0][0]']
ization)
conv4_block7_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block7_2_bn[0][0]']
n)
conv4_block7_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block7_2_relu[0][0]']
conv4_block7_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block7_3_conv[0][0]']
ization)
conv4_block7_add (Add) (None, 3, 3, 1024) 0 ['conv4_block6_out[0][0]',
'conv4_block7_3_bn[0][0]']
conv4_block7_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block7_add[0][0]']
conv4_block8_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block7_out[0][0]']
conv4_block8_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block8_1_conv[0][0]']
ization)
conv4_block8_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block8_1_bn[0][0]']
n)
conv4_block8_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block8_1_relu[0][0]']
conv4_block8_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block8_2_conv[0][0]']
ization)
conv4_block8_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block8_2_bn[0][0]']
n)
conv4_block8_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block8_2_relu[0][0]']
conv4_block8_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block8_3_conv[0][0]']
ization)
conv4_block8_add (Add) (None, 3, 3, 1024) 0 ['conv4_block7_out[0][0]',
'conv4_block8_3_bn[0][0]']
conv4_block8_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block8_add[0][0]']
conv4_block9_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block8_out[0][0]']
conv4_block9_1_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block9_1_conv[0][0]']
ization)
conv4_block9_1_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block9_1_bn[0][0]']
n)
conv4_block9_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block9_1_relu[0][0]']
conv4_block9_2_bn (BatchNormal (None, 3, 3, 256) 1024 ['conv4_block9_2_conv[0][0]']
ization)
conv4_block9_2_relu (Activatio (None, 3, 3, 256) 0 ['conv4_block9_2_bn[0][0]']
n)
conv4_block9_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block9_2_relu[0][0]']
conv4_block9_3_bn (BatchNormal (None, 3, 3, 1024) 4096 ['conv4_block9_3_conv[0][0]']
ization)
conv4_block9_add (Add) (None, 3, 3, 1024) 0 ['conv4_block8_out[0][0]',
'conv4_block9_3_bn[0][0]']
conv4_block9_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block9_add[0][0]']
conv4_block10_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block9_out[0][0]']
conv4_block10_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block10_1_conv[0][0]']
lization)
conv4_block10_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block10_1_bn[0][0]']
on)
conv4_block10_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block10_1_relu[0][0]']
conv4_block10_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block10_2_conv[0][0]']
lization)
conv4_block10_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block10_2_bn[0][0]']
on)
conv4_block10_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block10_2_relu[0][0]']
conv4_block10_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block10_3_conv[0][0]']
lization)
conv4_block10_add (Add) (None, 3, 3, 1024) 0 ['conv4_block9_out[0][0]',
'conv4_block10_3_bn[0][0]']
conv4_block10_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block10_add[0][0]']
conv4_block11_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block10_out[0][0]']
conv4_block11_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block11_1_conv[0][0]']
lization)
conv4_block11_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block11_1_bn[0][0]']
on)
conv4_block11_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block11_1_relu[0][0]']
conv4_block11_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block11_2_conv[0][0]']
lization)
conv4_block11_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block11_2_bn[0][0]']
on)
conv4_block11_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block11_2_relu[0][0]']
conv4_block11_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block11_3_conv[0][0]']
lization)
conv4_block11_add (Add) (None, 3, 3, 1024) 0 ['conv4_block10_out[0][0]',
'conv4_block11_3_bn[0][0]']
conv4_block11_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block11_add[0][0]']
conv4_block12_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block11_out[0][0]']
conv4_block12_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block12_1_conv[0][0]']
lization)
conv4_block12_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block12_1_bn[0][0]']
on)
conv4_block12_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block12_1_relu[0][0]']
conv4_block12_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block12_2_conv[0][0]']
lization)
conv4_block12_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block12_2_bn[0][0]']
on)
conv4_block12_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block12_2_relu[0][0]']
conv4_block12_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block12_3_conv[0][0]']
lization)
conv4_block12_add (Add) (None, 3, 3, 1024) 0 ['conv4_block11_out[0][0]',
'conv4_block12_3_bn[0][0]']
conv4_block12_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block12_add[0][0]']
conv4_block13_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block12_out[0][0]']
conv4_block13_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block13_1_conv[0][0]']
lization)
conv4_block13_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block13_1_bn[0][0]']
on)
conv4_block13_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block13_1_relu[0][0]']
conv4_block13_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block13_2_conv[0][0]']
lization)
conv4_block13_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block13_2_bn[0][0]']
on)
conv4_block13_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block13_2_relu[0][0]']
conv4_block13_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block13_3_conv[0][0]']
lization)
conv4_block13_add (Add) (None, 3, 3, 1024) 0 ['conv4_block12_out[0][0]',
'conv4_block13_3_bn[0][0]']
conv4_block13_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block13_add[0][0]']
conv4_block14_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block13_out[0][0]']
conv4_block14_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block14_1_conv[0][0]']
lization)
conv4_block14_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block14_1_bn[0][0]']
on)
conv4_block14_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block14_1_relu[0][0]']
conv4_block14_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block14_2_conv[0][0]']
lization)
conv4_block14_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block14_2_bn[0][0]']
on)
conv4_block14_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block14_2_relu[0][0]']
conv4_block14_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block14_3_conv[0][0]']
lization)
conv4_block14_add (Add) (None, 3, 3, 1024) 0 ['conv4_block13_out[0][0]',
'conv4_block14_3_bn[0][0]']
conv4_block14_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block14_add[0][0]']
conv4_block15_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block14_out[0][0]']
conv4_block15_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block15_1_conv[0][0]']
lization)
conv4_block15_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block15_1_bn[0][0]']
on)
conv4_block15_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block15_1_relu[0][0]']
conv4_block15_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block15_2_conv[0][0]']
lization)
conv4_block15_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block15_2_bn[0][0]']
on)
conv4_block15_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block15_2_relu[0][0]']
conv4_block15_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block15_3_conv[0][0]']
lization)
conv4_block15_add (Add) (None, 3, 3, 1024) 0 ['conv4_block14_out[0][0]',
'conv4_block15_3_bn[0][0]']
conv4_block15_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block15_add[0][0]']
conv4_block16_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block15_out[0][0]']
conv4_block16_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block16_1_conv[0][0]']
lization)
conv4_block16_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block16_1_bn[0][0]']
on)
conv4_block16_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block16_1_relu[0][0]']
conv4_block16_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block16_2_conv[0][0]']
lization)
conv4_block16_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block16_2_bn[0][0]']
on)
conv4_block16_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block16_2_relu[0][0]']
conv4_block16_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block16_3_conv[0][0]']
lization)
conv4_block16_add (Add) (None, 3, 3, 1024) 0 ['conv4_block15_out[0][0]',
'conv4_block16_3_bn[0][0]']
conv4_block16_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block16_add[0][0]']
conv4_block17_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block16_out[0][0]']
conv4_block17_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block17_1_conv[0][0]']
lization)
conv4_block17_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block17_1_bn[0][0]']
on)
conv4_block17_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block17_1_relu[0][0]']
conv4_block17_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block17_2_conv[0][0]']
lization)
conv4_block17_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block17_2_bn[0][0]']
on)
conv4_block17_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block17_2_relu[0][0]']
conv4_block17_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block17_3_conv[0][0]']
lization)
conv4_block17_add (Add) (None, 3, 3, 1024) 0 ['conv4_block16_out[0][0]',
'conv4_block17_3_bn[0][0]']
conv4_block17_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block17_add[0][0]']
conv4_block18_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block17_out[0][0]']
conv4_block18_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block18_1_conv[0][0]']
lization)
conv4_block18_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block18_1_bn[0][0]']
on)
conv4_block18_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block18_1_relu[0][0]']
conv4_block18_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block18_2_conv[0][0]']
lization)
conv4_block18_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block18_2_bn[0][0]']
on)
conv4_block18_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block18_2_relu[0][0]']
conv4_block18_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block18_3_conv[0][0]']
lization)
conv4_block18_add (Add) (None, 3, 3, 1024) 0 ['conv4_block17_out[0][0]',
'conv4_block18_3_bn[0][0]']
conv4_block18_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block18_add[0][0]']
conv4_block19_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block18_out[0][0]']
conv4_block19_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block19_1_conv[0][0]']
lization)
conv4_block19_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block19_1_bn[0][0]']
on)
conv4_block19_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block19_1_relu[0][0]']
conv4_block19_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block19_2_conv[0][0]']
lization)
conv4_block19_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block19_2_bn[0][0]']
on)
conv4_block19_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block19_2_relu[0][0]']
conv4_block19_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block19_3_conv[0][0]']
lization)
conv4_block19_add (Add) (None, 3, 3, 1024) 0 ['conv4_block18_out[0][0]',
'conv4_block19_3_bn[0][0]']
conv4_block19_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block19_add[0][0]']
conv4_block20_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block19_out[0][0]']
conv4_block20_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block20_1_conv[0][0]']
lization)
conv4_block20_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block20_1_bn[0][0]']
on)
conv4_block20_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block20_1_relu[0][0]']
conv4_block20_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block20_2_conv[0][0]']
lization)
conv4_block20_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block20_2_bn[0][0]']
on)
conv4_block20_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block20_2_relu[0][0]']
conv4_block20_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block20_3_conv[0][0]']
lization)
conv4_block20_add (Add) (None, 3, 3, 1024) 0 ['conv4_block19_out[0][0]',
'conv4_block20_3_bn[0][0]']
conv4_block20_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block20_add[0][0]']
conv4_block21_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block20_out[0][0]']
conv4_block21_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block21_1_conv[0][0]']
lization)
conv4_block21_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block21_1_bn[0][0]']
on)
conv4_block21_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block21_1_relu[0][0]']
conv4_block21_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block21_2_conv[0][0]']
lization)
conv4_block21_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block21_2_bn[0][0]']
on)
conv4_block21_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block21_2_relu[0][0]']
conv4_block21_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block21_3_conv[0][0]']
lization)
conv4_block21_add (Add) (None, 3, 3, 1024) 0 ['conv4_block20_out[0][0]',
'conv4_block21_3_bn[0][0]']
conv4_block21_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block21_add[0][0]']
conv4_block22_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block21_out[0][0]']
conv4_block22_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block22_1_conv[0][0]']
lization)
conv4_block22_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block22_1_bn[0][0]']
on)
conv4_block22_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block22_1_relu[0][0]']
conv4_block22_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block22_2_conv[0][0]']
lization)
conv4_block22_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block22_2_bn[0][0]']
on)
conv4_block22_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block22_2_relu[0][0]']
conv4_block22_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block22_3_conv[0][0]']
lization)
conv4_block22_add (Add) (None, 3, 3, 1024) 0 ['conv4_block21_out[0][0]',
'conv4_block22_3_bn[0][0]']
conv4_block22_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block22_add[0][0]']
conv4_block23_1_conv (Conv2D) (None, 3, 3, 256) 262400 ['conv4_block22_out[0][0]']
conv4_block23_1_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block23_1_conv[0][0]']
lization)
conv4_block23_1_relu (Activati (None, 3, 3, 256) 0 ['conv4_block23_1_bn[0][0]']
on)
conv4_block23_2_conv (Conv2D) (None, 3, 3, 256) 590080 ['conv4_block23_1_relu[0][0]']
conv4_block23_2_bn (BatchNorma (None, 3, 3, 256) 1024 ['conv4_block23_2_conv[0][0]']
lization)
conv4_block23_2_relu (Activati (None, 3, 3, 256) 0 ['conv4_block23_2_bn[0][0]']
on)
conv4_block23_3_conv (Conv2D) (None, 3, 3, 1024) 263168 ['conv4_block23_2_relu[0][0]']
conv4_block23_3_bn (BatchNorma (None, 3, 3, 1024) 4096 ['conv4_block23_3_conv[0][0]']
lization)
conv4_block23_add (Add) (None, 3, 3, 1024) 0 ['conv4_block22_out[0][0]',
'conv4_block23_3_bn[0][0]']
conv4_block23_out (Activation) (None, 3, 3, 1024) 0 ['conv4_block23_add[0][0]']
conv5_block1_1_conv (Conv2D) (None, 2, 2, 512) 524800 ['conv4_block23_out[0][0]']
conv5_block1_1_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block1_1_conv[0][0]']
ization)
conv5_block1_1_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block1_1_bn[0][0]']
n)
conv5_block1_2_conv (Conv2D) (None, 2, 2, 512) 2359808 ['conv5_block1_1_relu[0][0]']
conv5_block1_2_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block1_2_conv[0][0]']
ization)
conv5_block1_2_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block1_2_bn[0][0]']
n)
conv5_block1_0_conv (Conv2D) (None, 2, 2, 2048) 2099200 ['conv4_block23_out[0][0]']
conv5_block1_3_conv (Conv2D) (None, 2, 2, 2048) 1050624 ['conv5_block1_2_relu[0][0]']
conv5_block1_0_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block1_0_conv[0][0]']
ization)
conv5_block1_3_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block1_3_conv[0][0]']
ization)
conv5_block1_add (Add) (None, 2, 2, 2048) 0 ['conv5_block1_0_bn[0][0]',
'conv5_block1_3_bn[0][0]']
conv5_block1_out (Activation) (None, 2, 2, 2048) 0 ['conv5_block1_add[0][0]']
conv5_block2_1_conv (Conv2D) (None, 2, 2, 512) 1049088 ['conv5_block1_out[0][0]']
conv5_block2_1_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block2_1_conv[0][0]']
ization)
conv5_block2_1_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block2_1_bn[0][0]']
n)
conv5_block2_2_conv (Conv2D) (None, 2, 2, 512) 2359808 ['conv5_block2_1_relu[0][0]']
conv5_block2_2_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block2_2_conv[0][0]']
ization)
conv5_block2_2_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block2_2_bn[0][0]']
n)
conv5_block2_3_conv (Conv2D) (None, 2, 2, 2048) 1050624 ['conv5_block2_2_relu[0][0]']
conv5_block2_3_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block2_3_conv[0][0]']
ization)
conv5_block2_add (Add) (None, 2, 2, 2048) 0 ['conv5_block1_out[0][0]',
'conv5_block2_3_bn[0][0]']
conv5_block2_out (Activation) (None, 2, 2, 2048) 0 ['conv5_block2_add[0][0]']
conv5_block3_1_conv (Conv2D) (None, 2, 2, 512) 1049088 ['conv5_block2_out[0][0]']
conv5_block3_1_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block3_1_conv[0][0]']
ization)
conv5_block3_1_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block3_1_bn[0][0]']
n)
conv5_block3_2_conv (Conv2D) (None, 2, 2, 512) 2359808 ['conv5_block3_1_relu[0][0]']
conv5_block3_2_bn (BatchNormal (None, 2, 2, 512) 2048 ['conv5_block3_2_conv[0][0]']
ization)
conv5_block3_2_relu (Activatio (None, 2, 2, 512) 0 ['conv5_block3_2_bn[0][0]']
n)
conv5_block3_3_conv (Conv2D) (None, 2, 2, 2048) 1050624 ['conv5_block3_2_relu[0][0]']
conv5_block3_3_bn (BatchNormal (None, 2, 2, 2048) 8192 ['conv5_block3_3_conv[0][0]']
ization)
conv5_block3_add (Add) (None, 2, 2, 2048) 0 ['conv5_block2_out[0][0]',
'conv5_block3_3_bn[0][0]']
conv5_block3_out (Activation) (None, 2, 2, 2048) 0 ['conv5_block3_add[0][0]']
==================================================================================================
Total params: 42,658,176
Trainable params: 42,552,832
Non-trainable params: 105,344
__________________________________________________________________________________________________
# Model 4: ResNet V2
def model_builder_resnet(input_shape, tune=0):
"""Transfer approach with the ResNet V2
"""
resnet_model = ap.ResNet101(include_top=False, weights="imagenet", input_shape=input_shape)
# Making all the layers of the VGG model non-trainable
if tune > 0:
for layer in resnet_model.layers[:-tune]:
layer.trainable = False
else:
for layer in resnet_model.layers:
layer.trainable = False
model = Sequential([
resnet_model,
Flatten(),
Dense(256, activation='relu'),
Dense(128, activation='relu'),
Dropout(0.3),
Dense(64, activation='relu'),
BatchNormalization(),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_resnet, metric_resnet, history_resnet = model_building_and_evaluating_process(
model_builder_resnet, f'Model 4: Resnet V2 ({batch_size})', batch_size=batch_size, epochs=20, include_grayscale=False
)
---------------------------------------------
Model 4: Resnet V2 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
resnet101 (Functional) (None, 2, 2, 2048) 42658176
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 256) 2097408
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 44,797,252
Trainable params: 2,138,948
Non-trainable params: 42,658,304
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 73s 140ms/step - loss: 1.4625 - accuracy: 0.2616 - val_loss: 1.3638 - val_accuracy: 0.2447 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 62s 131ms/step - loss: 1.3958 - accuracy: 0.2665 - val_loss: 1.3371 - val_accuracy: 0.3378 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 58s 123ms/step - loss: 1.3655 - accuracy: 0.3060 - val_loss: 1.5133 - val_accuracy: 0.1836 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 58s 122ms/step - loss: 1.3194 - accuracy: 0.3426 - val_loss: 1.5810 - val_accuracy: 0.1943 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 59s 124ms/step - loss: 1.2924 - accuracy: 0.3683 - val_loss: 1.5175 - val_accuracy: 0.2504 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 60s 127ms/step - loss: 1.2399 - accuracy: 0.4201 - val_loss: 1.4819 - val_accuracy: 0.3183 - lr: 2.0000e-04
Epoch 7/20
473/473 [==============================] - 61s 129ms/step - loss: 1.2306 - accuracy: 0.4280 - val_loss: 1.2199 - val_accuracy: 0.4354 - lr: 2.0000e-04
Epoch 8/20
473/473 [==============================] - 60s 128ms/step - loss: 1.2215 - accuracy: 0.4339 - val_loss: 1.2098 - val_accuracy: 0.4493 - lr: 2.0000e-04
Epoch 9/20
473/473 [==============================] - 60s 127ms/step - loss: 1.2194 - accuracy: 0.4398 - val_loss: 1.2113 - val_accuracy: 0.4517 - lr: 2.0000e-04
Epoch 10/20
473/473 [==============================] - 58s 123ms/step - loss: 1.2133 - accuracy: 0.4459 - val_loss: 1.3069 - val_accuracy: 0.3908 - lr: 2.0000e-04
Epoch 11/20
473/473 [==============================] - 61s 128ms/step - loss: 1.2091 - accuracy: 0.4492 - val_loss: 1.1824 - val_accuracy: 0.4678 - lr: 2.0000e-04
Epoch 12/20
473/473 [==============================] - 59s 125ms/step - loss: 1.2051 - accuracy: 0.4526 - val_loss: 1.1814 - val_accuracy: 0.4671 - lr: 2.0000e-04
Epoch 13/20
473/473 [==============================] - 61s 128ms/step - loss: 1.2002 - accuracy: 0.4555 - val_loss: 1.1783 - val_accuracy: 0.4778 - lr: 2.0000e-04
Epoch 14/20
473/473 [==============================] - 58s 122ms/step - loss: 1.1955 - accuracy: 0.4554 - val_loss: 1.1905 - val_accuracy: 0.4669 - lr: 2.0000e-04
Epoch 15/20
473/473 [==============================] - 58s 122ms/step - loss: 1.1942 - accuracy: 0.4567 - val_loss: 1.3811 - val_accuracy: 0.3641 - lr: 2.0000e-04
Epoch 16/20
473/473 [==============================] - 58s 123ms/step - loss: 1.1909 - accuracy: 0.4618 - val_loss: 1.1910 - val_accuracy: 0.4736 - lr: 2.0000e-04
Epoch 17/20
473/473 [==============================] - 58s 124ms/step - loss: 1.1796 - accuracy: 0.4757 - val_loss: 1.1884 - val_accuracy: 0.4643 - lr: 4.0000e-05
Epoch 18/20
473/473 [==============================] - 58s 122ms/step - loss: 1.1780 - accuracy: 0.4677 - val_loss: 1.2073 - val_accuracy: 0.4541 - lr: 4.0000e-05
1/1 [==============================] - 0s 111ms/step - loss: 1.1514 - accuracy: 0.5312
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 2 1 3 2 1 0 2 2 2 2 2 0 0 3 1 0 0 2 2 1 1 0 2 0 0 2 0 0 0]
precision recall f1-score support
happy 0.58 0.70 0.64 10
neutral 0.50 0.60 0.55 5
sad 0.36 0.40 0.38 10
surprise 1.00 0.43 0.60 7
accuracy 0.53 32
macro avg 0.61 0.53 0.54 32
weighted avg 0.59 0.53 0.53 32
# Let's try again tuning some of the last layers in the transfer model
batch_size = 32
model, results, history = model_building_and_evaluating_process(model_builder_resnet, f'Model 4: Tuned Resnet V2 ({batch_size})',
batch_size=batch_size, epochs=20, tune=5, include_grayscale=False)
model_resnet.update(model)
metric_resnet = pd.concat([metric_resnet, results])
history_resnet.update(history)
---------------------------------------------
Model 4: Tuned Resnet V2 (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
resnet101 (Functional) (None, 2, 2, 2048) 42658176
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 256) 2097408
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 44,797,252
Trainable params: 3,193,668
Non-trainable params: 41,603,584
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 72s 136ms/step - loss: 1.4439 - accuracy: 0.2798 - val_loss: 1.3643 - val_accuracy: 0.2339 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 62s 131ms/step - loss: 1.3400 - accuracy: 0.3270 - val_loss: 1.4209 - val_accuracy: 0.2522 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 62s 131ms/step - loss: 1.3090 - accuracy: 0.3579 - val_loss: 1.3305 - val_accuracy: 0.3390 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 60s 126ms/step - loss: 1.2867 - accuracy: 0.3843 - val_loss: 1.4322 - val_accuracy: 0.2518 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 59s 125ms/step - loss: 1.2718 - accuracy: 0.3998 - val_loss: 1.3774 - val_accuracy: 0.3138 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 62s 132ms/step - loss: 1.2544 - accuracy: 0.4141 - val_loss: 1.3111 - val_accuracy: 0.3659 - lr: 0.0010
Epoch 7/20
473/473 [==============================] - 62s 132ms/step - loss: 1.2385 - accuracy: 0.4211 - val_loss: 1.2145 - val_accuracy: 0.4400 - lr: 0.0010
Epoch 8/20
473/473 [==============================] - 61s 128ms/step - loss: 1.2297 - accuracy: 0.4289 - val_loss: 1.3236 - val_accuracy: 0.3500 - lr: 0.0010
Epoch 9/20
473/473 [==============================] - 60s 127ms/step - loss: 1.2218 - accuracy: 0.4342 - val_loss: 1.5070 - val_accuracy: 0.3516 - lr: 0.0010
Epoch 10/20
473/473 [==============================] - 62s 132ms/step - loss: 1.2037 - accuracy: 0.4498 - val_loss: 1.1793 - val_accuracy: 0.4601 - lr: 0.0010
Epoch 11/20
473/473 [==============================] - 62s 131ms/step - loss: 1.1928 - accuracy: 0.4614 - val_loss: 1.2029 - val_accuracy: 0.4641 - lr: 0.0010
Epoch 12/20
473/473 [==============================] - 60s 128ms/step - loss: 1.1891 - accuracy: 0.4591 - val_loss: 1.4216 - val_accuracy: 0.4089 - lr: 0.0010
Epoch 13/20
473/473 [==============================] - 60s 127ms/step - loss: 1.1772 - accuracy: 0.4688 - val_loss: 1.4118 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 14/20
473/473 [==============================] - 62s 131ms/step - loss: 1.1333 - accuracy: 0.5008 - val_loss: 1.1229 - val_accuracy: 0.4967 - lr: 2.0000e-04
Epoch 15/20
473/473 [==============================] - 62s 131ms/step - loss: 1.1315 - accuracy: 0.5072 - val_loss: 1.1263 - val_accuracy: 0.5011 - lr: 2.0000e-04
Epoch 16/20
473/473 [==============================] - 60s 127ms/step - loss: 1.1244 - accuracy: 0.5091 - val_loss: 1.1475 - val_accuracy: 0.4895 - lr: 2.0000e-04
Epoch 17/20
473/473 [==============================] - 62s 130ms/step - loss: 1.1192 - accuracy: 0.5137 - val_loss: 1.1321 - val_accuracy: 0.5067 - lr: 2.0000e-04
Epoch 18/20
473/473 [==============================] - 60s 127ms/step - loss: 1.1115 - accuracy: 0.5130 - val_loss: 1.1718 - val_accuracy: 0.4657 - lr: 4.0000e-05
Epoch 19/20
473/473 [==============================] - 59s 125ms/step - loss: 1.1112 - accuracy: 0.5166 - val_loss: 1.1363 - val_accuracy: 0.4999 - lr: 4.0000e-05
1/1 [==============================] - 0s 113ms/step - loss: 1.1130 - accuracy: 0.3750
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 0 3 1 1 0 1 1 0 3 2 2 2 0 1 0 3 1 0 0 0 2 1 1 0 2 0 1 1 0 0 0]
precision recall f1-score support
happy 0.43 0.60 0.50 10
neutral 0.30 0.60 0.40 5
sad 0.20 0.10 0.13 10
surprise 0.67 0.29 0.40 7
accuracy 0.38 32
macro avg 0.40 0.40 0.36 32
weighted avg 0.39 0.38 0.35 32
model_name = 'Model 4: Resnet V2'
save_object(history_resnet, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_resnet, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_resnet, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_resnet, data_type='Training Set')
loss_and_accuracy_comparisson(history_resnet, data_type='Validation Set')
metric_resnet.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 4: Resnet V2 (32) - RGB | 0.455490 | 0.477798 | 0.531250 |
| Model 4: Tuned Resnet V2 (32) - RGB | 0.513667 | 0.506731 | 0.375000 |
Note: You can even go back and build your own architecture on top of the ResNet Transfer layer and see if you can improve the performance.
⏩ Observations and Insights:
⏩ Recommendations:
ap.EfficientNetV2B2(include_top=False, weights="imagenet", input_shape=(48, 48, 3)).summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/efficientnet_v2/efficientnetv2-b2_notop.h5
35839040/35839040 [==============================] - 1s 0us/step
Model: "efficientnetv2-b2"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 48, 48, 3)] 0 []
rescaling (Rescaling) (None, 48, 48, 3) 0 ['input_2[0][0]']
normalization (Normalization) (None, 48, 48, 3) 0 ['rescaling[0][0]']
stem_conv (Conv2D) (None, 24, 24, 32) 864 ['normalization[0][0]']
stem_bn (BatchNormalization) (None, 24, 24, 32) 128 ['stem_conv[0][0]']
stem_activation (Activation) (None, 24, 24, 32) 0 ['stem_bn[0][0]']
block1a_project_conv (Conv2D) (None, 24, 24, 16) 4608 ['stem_activation[0][0]']
block1a_project_bn (BatchNorma (None, 24, 24, 16) 64 ['block1a_project_conv[0][0]']
lization)
block1a_project_activation (Ac (None, 24, 24, 16) 0 ['block1a_project_bn[0][0]']
tivation)
block1b_project_conv (Conv2D) (None, 24, 24, 16) 2304 ['block1a_project_activation[0][0
]']
block1b_project_bn (BatchNorma (None, 24, 24, 16) 64 ['block1b_project_conv[0][0]']
lization)
block1b_project_activation (Ac (None, 24, 24, 16) 0 ['block1b_project_bn[0][0]']
tivation)
block1b_drop (Dropout) (None, 24, 24, 16) 0 ['block1b_project_activation[0][0
]']
block1b_add (Add) (None, 24, 24, 16) 0 ['block1b_drop[0][0]',
'block1a_project_activation[0][0
]']
block2a_expand_conv (Conv2D) (None, 12, 12, 64) 9216 ['block1b_add[0][0]']
block2a_expand_bn (BatchNormal (None, 12, 12, 64) 256 ['block2a_expand_conv[0][0]']
ization)
block2a_expand_activation (Act (None, 12, 12, 64) 0 ['block2a_expand_bn[0][0]']
ivation)
block2a_project_conv (Conv2D) (None, 12, 12, 32) 2048 ['block2a_expand_activation[0][0]
']
block2a_project_bn (BatchNorma (None, 12, 12, 32) 128 ['block2a_project_conv[0][0]']
lization)
block2b_expand_conv (Conv2D) (None, 12, 12, 128) 36864 ['block2a_project_bn[0][0]']
block2b_expand_bn (BatchNormal (None, 12, 12, 128) 512 ['block2b_expand_conv[0][0]']
ization)
block2b_expand_activation (Act (None, 12, 12, 128) 0 ['block2b_expand_bn[0][0]']
ivation)
block2b_project_conv (Conv2D) (None, 12, 12, 32) 4096 ['block2b_expand_activation[0][0]
']
block2b_project_bn (BatchNorma (None, 12, 12, 32) 128 ['block2b_project_conv[0][0]']
lization)
block2b_drop (Dropout) (None, 12, 12, 32) 0 ['block2b_project_bn[0][0]']
block2b_add (Add) (None, 12, 12, 32) 0 ['block2b_drop[0][0]',
'block2a_project_bn[0][0]']
block2c_expand_conv (Conv2D) (None, 12, 12, 128) 36864 ['block2b_add[0][0]']
block2c_expand_bn (BatchNormal (None, 12, 12, 128) 512 ['block2c_expand_conv[0][0]']
ization)
block2c_expand_activation (Act (None, 12, 12, 128) 0 ['block2c_expand_bn[0][0]']
ivation)
block2c_project_conv (Conv2D) (None, 12, 12, 32) 4096 ['block2c_expand_activation[0][0]
']
block2c_project_bn (BatchNorma (None, 12, 12, 32) 128 ['block2c_project_conv[0][0]']
lization)
block2c_drop (Dropout) (None, 12, 12, 32) 0 ['block2c_project_bn[0][0]']
block2c_add (Add) (None, 12, 12, 32) 0 ['block2c_drop[0][0]',
'block2b_add[0][0]']
block3a_expand_conv (Conv2D) (None, 6, 6, 128) 36864 ['block2c_add[0][0]']
block3a_expand_bn (BatchNormal (None, 6, 6, 128) 512 ['block3a_expand_conv[0][0]']
ization)
block3a_expand_activation (Act (None, 6, 6, 128) 0 ['block3a_expand_bn[0][0]']
ivation)
block3a_project_conv (Conv2D) (None, 6, 6, 56) 7168 ['block3a_expand_activation[0][0]
']
block3a_project_bn (BatchNorma (None, 6, 6, 56) 224 ['block3a_project_conv[0][0]']
lization)
block3b_expand_conv (Conv2D) (None, 6, 6, 224) 112896 ['block3a_project_bn[0][0]']
block3b_expand_bn (BatchNormal (None, 6, 6, 224) 896 ['block3b_expand_conv[0][0]']
ization)
block3b_expand_activation (Act (None, 6, 6, 224) 0 ['block3b_expand_bn[0][0]']
ivation)
block3b_project_conv (Conv2D) (None, 6, 6, 56) 12544 ['block3b_expand_activation[0][0]
']
block3b_project_bn (BatchNorma (None, 6, 6, 56) 224 ['block3b_project_conv[0][0]']
lization)
block3b_drop (Dropout) (None, 6, 6, 56) 0 ['block3b_project_bn[0][0]']
block3b_add (Add) (None, 6, 6, 56) 0 ['block3b_drop[0][0]',
'block3a_project_bn[0][0]']
block3c_expand_conv (Conv2D) (None, 6, 6, 224) 112896 ['block3b_add[0][0]']
block3c_expand_bn (BatchNormal (None, 6, 6, 224) 896 ['block3c_expand_conv[0][0]']
ization)
block3c_expand_activation (Act (None, 6, 6, 224) 0 ['block3c_expand_bn[0][0]']
ivation)
block3c_project_conv (Conv2D) (None, 6, 6, 56) 12544 ['block3c_expand_activation[0][0]
']
block3c_project_bn (BatchNorma (None, 6, 6, 56) 224 ['block3c_project_conv[0][0]']
lization)
block3c_drop (Dropout) (None, 6, 6, 56) 0 ['block3c_project_bn[0][0]']
block3c_add (Add) (None, 6, 6, 56) 0 ['block3c_drop[0][0]',
'block3b_add[0][0]']
block4a_expand_conv (Conv2D) (None, 6, 6, 224) 12544 ['block3c_add[0][0]']
block4a_expand_bn (BatchNormal (None, 6, 6, 224) 896 ['block4a_expand_conv[0][0]']
ization)
block4a_expand_activation (Act (None, 6, 6, 224) 0 ['block4a_expand_bn[0][0]']
ivation)
block4a_dwconv2 (DepthwiseConv (None, 3, 3, 224) 2016 ['block4a_expand_activation[0][0]
2D) ']
block4a_bn (BatchNormalization (None, 3, 3, 224) 896 ['block4a_dwconv2[0][0]']
)
block4a_activation (Activation (None, 3, 3, 224) 0 ['block4a_bn[0][0]']
)
block4a_se_squeeze (GlobalAver (None, 224) 0 ['block4a_activation[0][0]']
agePooling2D)
block4a_se_reshape (Reshape) (None, 1, 1, 224) 0 ['block4a_se_squeeze[0][0]']
block4a_se_reduce (Conv2D) (None, 1, 1, 14) 3150 ['block4a_se_reshape[0][0]']
block4a_se_expand (Conv2D) (None, 1, 1, 224) 3360 ['block4a_se_reduce[0][0]']
block4a_se_excite (Multiply) (None, 3, 3, 224) 0 ['block4a_activation[0][0]',
'block4a_se_expand[0][0]']
block4a_project_conv (Conv2D) (None, 3, 3, 104) 23296 ['block4a_se_excite[0][0]']
block4a_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4a_project_conv[0][0]']
lization)
block4b_expand_conv (Conv2D) (None, 3, 3, 416) 43264 ['block4a_project_bn[0][0]']
block4b_expand_bn (BatchNormal (None, 3, 3, 416) 1664 ['block4b_expand_conv[0][0]']
ization)
block4b_expand_activation (Act (None, 3, 3, 416) 0 ['block4b_expand_bn[0][0]']
ivation)
block4b_dwconv2 (DepthwiseConv (None, 3, 3, 416) 3744 ['block4b_expand_activation[0][0]
2D) ']
block4b_bn (BatchNormalization (None, 3, 3, 416) 1664 ['block4b_dwconv2[0][0]']
)
block4b_activation (Activation (None, 3, 3, 416) 0 ['block4b_bn[0][0]']
)
block4b_se_squeeze (GlobalAver (None, 416) 0 ['block4b_activation[0][0]']
agePooling2D)
block4b_se_reshape (Reshape) (None, 1, 1, 416) 0 ['block4b_se_squeeze[0][0]']
block4b_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4b_se_reshape[0][0]']
block4b_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4b_se_reduce[0][0]']
block4b_se_excite (Multiply) (None, 3, 3, 416) 0 ['block4b_activation[0][0]',
'block4b_se_expand[0][0]']
block4b_project_conv (Conv2D) (None, 3, 3, 104) 43264 ['block4b_se_excite[0][0]']
block4b_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4b_project_conv[0][0]']
lization)
block4b_drop (Dropout) (None, 3, 3, 104) 0 ['block4b_project_bn[0][0]']
block4b_add (Add) (None, 3, 3, 104) 0 ['block4b_drop[0][0]',
'block4a_project_bn[0][0]']
block4c_expand_conv (Conv2D) (None, 3, 3, 416) 43264 ['block4b_add[0][0]']
block4c_expand_bn (BatchNormal (None, 3, 3, 416) 1664 ['block4c_expand_conv[0][0]']
ization)
block4c_expand_activation (Act (None, 3, 3, 416) 0 ['block4c_expand_bn[0][0]']
ivation)
block4c_dwconv2 (DepthwiseConv (None, 3, 3, 416) 3744 ['block4c_expand_activation[0][0]
2D) ']
block4c_bn (BatchNormalization (None, 3, 3, 416) 1664 ['block4c_dwconv2[0][0]']
)
block4c_activation (Activation (None, 3, 3, 416) 0 ['block4c_bn[0][0]']
)
block4c_se_squeeze (GlobalAver (None, 416) 0 ['block4c_activation[0][0]']
agePooling2D)
block4c_se_reshape (Reshape) (None, 1, 1, 416) 0 ['block4c_se_squeeze[0][0]']
block4c_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4c_se_reshape[0][0]']
block4c_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4c_se_reduce[0][0]']
block4c_se_excite (Multiply) (None, 3, 3, 416) 0 ['block4c_activation[0][0]',
'block4c_se_expand[0][0]']
block4c_project_conv (Conv2D) (None, 3, 3, 104) 43264 ['block4c_se_excite[0][0]']
block4c_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4c_project_conv[0][0]']
lization)
block4c_drop (Dropout) (None, 3, 3, 104) 0 ['block4c_project_bn[0][0]']
block4c_add (Add) (None, 3, 3, 104) 0 ['block4c_drop[0][0]',
'block4b_add[0][0]']
block4d_expand_conv (Conv2D) (None, 3, 3, 416) 43264 ['block4c_add[0][0]']
block4d_expand_bn (BatchNormal (None, 3, 3, 416) 1664 ['block4d_expand_conv[0][0]']
ization)
block4d_expand_activation (Act (None, 3, 3, 416) 0 ['block4d_expand_bn[0][0]']
ivation)
block4d_dwconv2 (DepthwiseConv (None, 3, 3, 416) 3744 ['block4d_expand_activation[0][0]
2D) ']
block4d_bn (BatchNormalization (None, 3, 3, 416) 1664 ['block4d_dwconv2[0][0]']
)
block4d_activation (Activation (None, 3, 3, 416) 0 ['block4d_bn[0][0]']
)
block4d_se_squeeze (GlobalAver (None, 416) 0 ['block4d_activation[0][0]']
agePooling2D)
block4d_se_reshape (Reshape) (None, 1, 1, 416) 0 ['block4d_se_squeeze[0][0]']
block4d_se_reduce (Conv2D) (None, 1, 1, 26) 10842 ['block4d_se_reshape[0][0]']
block4d_se_expand (Conv2D) (None, 1, 1, 416) 11232 ['block4d_se_reduce[0][0]']
block4d_se_excite (Multiply) (None, 3, 3, 416) 0 ['block4d_activation[0][0]',
'block4d_se_expand[0][0]']
block4d_project_conv (Conv2D) (None, 3, 3, 104) 43264 ['block4d_se_excite[0][0]']
block4d_project_bn (BatchNorma (None, 3, 3, 104) 416 ['block4d_project_conv[0][0]']
lization)
block4d_drop (Dropout) (None, 3, 3, 104) 0 ['block4d_project_bn[0][0]']
block4d_add (Add) (None, 3, 3, 104) 0 ['block4d_drop[0][0]',
'block4c_add[0][0]']
block5a_expand_conv (Conv2D) (None, 3, 3, 624) 64896 ['block4d_add[0][0]']
block5a_expand_bn (BatchNormal (None, 3, 3, 624) 2496 ['block5a_expand_conv[0][0]']
ization)
block5a_expand_activation (Act (None, 3, 3, 624) 0 ['block5a_expand_bn[0][0]']
ivation)
block5a_dwconv2 (DepthwiseConv (None, 3, 3, 624) 5616 ['block5a_expand_activation[0][0]
2D) ']
block5a_bn (BatchNormalization (None, 3, 3, 624) 2496 ['block5a_dwconv2[0][0]']
)
block5a_activation (Activation (None, 3, 3, 624) 0 ['block5a_bn[0][0]']
)
block5a_se_squeeze (GlobalAver (None, 624) 0 ['block5a_activation[0][0]']
agePooling2D)
block5a_se_reshape (Reshape) (None, 1, 1, 624) 0 ['block5a_se_squeeze[0][0]']
block5a_se_reduce (Conv2D) (None, 1, 1, 26) 16250 ['block5a_se_reshape[0][0]']
block5a_se_expand (Conv2D) (None, 1, 1, 624) 16848 ['block5a_se_reduce[0][0]']
block5a_se_excite (Multiply) (None, 3, 3, 624) 0 ['block5a_activation[0][0]',
'block5a_se_expand[0][0]']
block5a_project_conv (Conv2D) (None, 3, 3, 120) 74880 ['block5a_se_excite[0][0]']
block5a_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5a_project_conv[0][0]']
lization)
block5b_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5a_project_bn[0][0]']
block5b_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5b_expand_conv[0][0]']
ization)
block5b_expand_activation (Act (None, 3, 3, 720) 0 ['block5b_expand_bn[0][0]']
ivation)
block5b_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5b_expand_activation[0][0]
2D) ']
block5b_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5b_dwconv2[0][0]']
)
block5b_activation (Activation (None, 3, 3, 720) 0 ['block5b_bn[0][0]']
)
block5b_se_squeeze (GlobalAver (None, 720) 0 ['block5b_activation[0][0]']
agePooling2D)
block5b_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5b_se_squeeze[0][0]']
block5b_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5b_se_reshape[0][0]']
block5b_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5b_se_reduce[0][0]']
block5b_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5b_activation[0][0]',
'block5b_se_expand[0][0]']
block5b_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5b_se_excite[0][0]']
block5b_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5b_project_conv[0][0]']
lization)
block5b_drop (Dropout) (None, 3, 3, 120) 0 ['block5b_project_bn[0][0]']
block5b_add (Add) (None, 3, 3, 120) 0 ['block5b_drop[0][0]',
'block5a_project_bn[0][0]']
block5c_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5b_add[0][0]']
block5c_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5c_expand_conv[0][0]']
ization)
block5c_expand_activation (Act (None, 3, 3, 720) 0 ['block5c_expand_bn[0][0]']
ivation)
block5c_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5c_expand_activation[0][0]
2D) ']
block5c_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5c_dwconv2[0][0]']
)
block5c_activation (Activation (None, 3, 3, 720) 0 ['block5c_bn[0][0]']
)
block5c_se_squeeze (GlobalAver (None, 720) 0 ['block5c_activation[0][0]']
agePooling2D)
block5c_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5c_se_squeeze[0][0]']
block5c_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5c_se_reshape[0][0]']
block5c_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5c_se_reduce[0][0]']
block5c_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5c_activation[0][0]',
'block5c_se_expand[0][0]']
block5c_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5c_se_excite[0][0]']
block5c_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5c_project_conv[0][0]']
lization)
block5c_drop (Dropout) (None, 3, 3, 120) 0 ['block5c_project_bn[0][0]']
block5c_add (Add) (None, 3, 3, 120) 0 ['block5c_drop[0][0]',
'block5b_add[0][0]']
block5d_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5c_add[0][0]']
block5d_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5d_expand_conv[0][0]']
ization)
block5d_expand_activation (Act (None, 3, 3, 720) 0 ['block5d_expand_bn[0][0]']
ivation)
block5d_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5d_expand_activation[0][0]
2D) ']
block5d_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5d_dwconv2[0][0]']
)
block5d_activation (Activation (None, 3, 3, 720) 0 ['block5d_bn[0][0]']
)
block5d_se_squeeze (GlobalAver (None, 720) 0 ['block5d_activation[0][0]']
agePooling2D)
block5d_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5d_se_squeeze[0][0]']
block5d_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5d_se_reshape[0][0]']
block5d_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5d_se_reduce[0][0]']
block5d_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5d_activation[0][0]',
'block5d_se_expand[0][0]']
block5d_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5d_se_excite[0][0]']
block5d_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5d_project_conv[0][0]']
lization)
block5d_drop (Dropout) (None, 3, 3, 120) 0 ['block5d_project_bn[0][0]']
block5d_add (Add) (None, 3, 3, 120) 0 ['block5d_drop[0][0]',
'block5c_add[0][0]']
block5e_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5d_add[0][0]']
block5e_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5e_expand_conv[0][0]']
ization)
block5e_expand_activation (Act (None, 3, 3, 720) 0 ['block5e_expand_bn[0][0]']
ivation)
block5e_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5e_expand_activation[0][0]
2D) ']
block5e_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5e_dwconv2[0][0]']
)
block5e_activation (Activation (None, 3, 3, 720) 0 ['block5e_bn[0][0]']
)
block5e_se_squeeze (GlobalAver (None, 720) 0 ['block5e_activation[0][0]']
agePooling2D)
block5e_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5e_se_squeeze[0][0]']
block5e_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5e_se_reshape[0][0]']
block5e_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5e_se_reduce[0][0]']
block5e_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5e_activation[0][0]',
'block5e_se_expand[0][0]']
block5e_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5e_se_excite[0][0]']
block5e_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5e_project_conv[0][0]']
lization)
block5e_drop (Dropout) (None, 3, 3, 120) 0 ['block5e_project_bn[0][0]']
block5e_add (Add) (None, 3, 3, 120) 0 ['block5e_drop[0][0]',
'block5d_add[0][0]']
block5f_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5e_add[0][0]']
block5f_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block5f_expand_conv[0][0]']
ization)
block5f_expand_activation (Act (None, 3, 3, 720) 0 ['block5f_expand_bn[0][0]']
ivation)
block5f_dwconv2 (DepthwiseConv (None, 3, 3, 720) 6480 ['block5f_expand_activation[0][0]
2D) ']
block5f_bn (BatchNormalization (None, 3, 3, 720) 2880 ['block5f_dwconv2[0][0]']
)
block5f_activation (Activation (None, 3, 3, 720) 0 ['block5f_bn[0][0]']
)
block5f_se_squeeze (GlobalAver (None, 720) 0 ['block5f_activation[0][0]']
agePooling2D)
block5f_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block5f_se_squeeze[0][0]']
block5f_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block5f_se_reshape[0][0]']
block5f_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block5f_se_reduce[0][0]']
block5f_se_excite (Multiply) (None, 3, 3, 720) 0 ['block5f_activation[0][0]',
'block5f_se_expand[0][0]']
block5f_project_conv (Conv2D) (None, 3, 3, 120) 86400 ['block5f_se_excite[0][0]']
block5f_project_bn (BatchNorma (None, 3, 3, 120) 480 ['block5f_project_conv[0][0]']
lization)
block5f_drop (Dropout) (None, 3, 3, 120) 0 ['block5f_project_bn[0][0]']
block5f_add (Add) (None, 3, 3, 120) 0 ['block5f_drop[0][0]',
'block5e_add[0][0]']
block6a_expand_conv (Conv2D) (None, 3, 3, 720) 86400 ['block5f_add[0][0]']
block6a_expand_bn (BatchNormal (None, 3, 3, 720) 2880 ['block6a_expand_conv[0][0]']
ization)
block6a_expand_activation (Act (None, 3, 3, 720) 0 ['block6a_expand_bn[0][0]']
ivation)
block6a_dwconv2 (DepthwiseConv (None, 2, 2, 720) 6480 ['block6a_expand_activation[0][0]
2D) ']
block6a_bn (BatchNormalization (None, 2, 2, 720) 2880 ['block6a_dwconv2[0][0]']
)
block6a_activation (Activation (None, 2, 2, 720) 0 ['block6a_bn[0][0]']
)
block6a_se_squeeze (GlobalAver (None, 720) 0 ['block6a_activation[0][0]']
agePooling2D)
block6a_se_reshape (Reshape) (None, 1, 1, 720) 0 ['block6a_se_squeeze[0][0]']
block6a_se_reduce (Conv2D) (None, 1, 1, 30) 21630 ['block6a_se_reshape[0][0]']
block6a_se_expand (Conv2D) (None, 1, 1, 720) 22320 ['block6a_se_reduce[0][0]']
block6a_se_excite (Multiply) (None, 2, 2, 720) 0 ['block6a_activation[0][0]',
'block6a_se_expand[0][0]']
block6a_project_conv (Conv2D) (None, 2, 2, 208) 149760 ['block6a_se_excite[0][0]']
block6a_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6a_project_conv[0][0]']
lization)
block6b_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6a_project_bn[0][0]']
block6b_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6b_expand_conv[0][0]']
ization)
block6b_expand_activation (Act (None, 2, 2, 1248) 0 ['block6b_expand_bn[0][0]']
ivation)
block6b_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6b_expand_activation[0][0]
2D) ']
block6b_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6b_dwconv2[0][0]']
)
block6b_activation (Activation (None, 2, 2, 1248) 0 ['block6b_bn[0][0]']
)
block6b_se_squeeze (GlobalAver (None, 1248) 0 ['block6b_activation[0][0]']
agePooling2D)
block6b_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6b_se_squeeze[0][0]']
block6b_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6b_se_reshape[0][0]']
block6b_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6b_se_reduce[0][0]']
block6b_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6b_activation[0][0]',
'block6b_se_expand[0][0]']
block6b_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6b_se_excite[0][0]']
block6b_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6b_project_conv[0][0]']
lization)
block6b_drop (Dropout) (None, 2, 2, 208) 0 ['block6b_project_bn[0][0]']
block6b_add (Add) (None, 2, 2, 208) 0 ['block6b_drop[0][0]',
'block6a_project_bn[0][0]']
block6c_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6b_add[0][0]']
block6c_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6c_expand_conv[0][0]']
ization)
block6c_expand_activation (Act (None, 2, 2, 1248) 0 ['block6c_expand_bn[0][0]']
ivation)
block6c_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6c_expand_activation[0][0]
2D) ']
block6c_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6c_dwconv2[0][0]']
)
block6c_activation (Activation (None, 2, 2, 1248) 0 ['block6c_bn[0][0]']
)
block6c_se_squeeze (GlobalAver (None, 1248) 0 ['block6c_activation[0][0]']
agePooling2D)
block6c_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6c_se_squeeze[0][0]']
block6c_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6c_se_reshape[0][0]']
block6c_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6c_se_reduce[0][0]']
block6c_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6c_activation[0][0]',
'block6c_se_expand[0][0]']
block6c_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6c_se_excite[0][0]']
block6c_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6c_project_conv[0][0]']
lization)
block6c_drop (Dropout) (None, 2, 2, 208) 0 ['block6c_project_bn[0][0]']
block6c_add (Add) (None, 2, 2, 208) 0 ['block6c_drop[0][0]',
'block6b_add[0][0]']
block6d_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6c_add[0][0]']
block6d_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6d_expand_conv[0][0]']
ization)
block6d_expand_activation (Act (None, 2, 2, 1248) 0 ['block6d_expand_bn[0][0]']
ivation)
block6d_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6d_expand_activation[0][0]
2D) ']
block6d_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6d_dwconv2[0][0]']
)
block6d_activation (Activation (None, 2, 2, 1248) 0 ['block6d_bn[0][0]']
)
block6d_se_squeeze (GlobalAver (None, 1248) 0 ['block6d_activation[0][0]']
agePooling2D)
block6d_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6d_se_squeeze[0][0]']
block6d_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6d_se_reshape[0][0]']
block6d_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6d_se_reduce[0][0]']
block6d_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6d_activation[0][0]',
'block6d_se_expand[0][0]']
block6d_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6d_se_excite[0][0]']
block6d_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6d_project_conv[0][0]']
lization)
block6d_drop (Dropout) (None, 2, 2, 208) 0 ['block6d_project_bn[0][0]']
block6d_add (Add) (None, 2, 2, 208) 0 ['block6d_drop[0][0]',
'block6c_add[0][0]']
block6e_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6d_add[0][0]']
block6e_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6e_expand_conv[0][0]']
ization)
block6e_expand_activation (Act (None, 2, 2, 1248) 0 ['block6e_expand_bn[0][0]']
ivation)
block6e_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6e_expand_activation[0][0]
2D) ']
block6e_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6e_dwconv2[0][0]']
)
block6e_activation (Activation (None, 2, 2, 1248) 0 ['block6e_bn[0][0]']
)
block6e_se_squeeze (GlobalAver (None, 1248) 0 ['block6e_activation[0][0]']
agePooling2D)
block6e_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6e_se_squeeze[0][0]']
block6e_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6e_se_reshape[0][0]']
block6e_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6e_se_reduce[0][0]']
block6e_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6e_activation[0][0]',
'block6e_se_expand[0][0]']
block6e_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6e_se_excite[0][0]']
block6e_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6e_project_conv[0][0]']
lization)
block6e_drop (Dropout) (None, 2, 2, 208) 0 ['block6e_project_bn[0][0]']
block6e_add (Add) (None, 2, 2, 208) 0 ['block6e_drop[0][0]',
'block6d_add[0][0]']
block6f_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6e_add[0][0]']
block6f_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6f_expand_conv[0][0]']
ization)
block6f_expand_activation (Act (None, 2, 2, 1248) 0 ['block6f_expand_bn[0][0]']
ivation)
block6f_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6f_expand_activation[0][0]
2D) ']
block6f_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6f_dwconv2[0][0]']
)
block6f_activation (Activation (None, 2, 2, 1248) 0 ['block6f_bn[0][0]']
)
block6f_se_squeeze (GlobalAver (None, 1248) 0 ['block6f_activation[0][0]']
agePooling2D)
block6f_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6f_se_squeeze[0][0]']
block6f_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6f_se_reshape[0][0]']
block6f_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6f_se_reduce[0][0]']
block6f_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6f_activation[0][0]',
'block6f_se_expand[0][0]']
block6f_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6f_se_excite[0][0]']
block6f_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6f_project_conv[0][0]']
lization)
block6f_drop (Dropout) (None, 2, 2, 208) 0 ['block6f_project_bn[0][0]']
block6f_add (Add) (None, 2, 2, 208) 0 ['block6f_drop[0][0]',
'block6e_add[0][0]']
block6g_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6f_add[0][0]']
block6g_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6g_expand_conv[0][0]']
ization)
block6g_expand_activation (Act (None, 2, 2, 1248) 0 ['block6g_expand_bn[0][0]']
ivation)
block6g_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6g_expand_activation[0][0]
2D) ']
block6g_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6g_dwconv2[0][0]']
)
block6g_activation (Activation (None, 2, 2, 1248) 0 ['block6g_bn[0][0]']
)
block6g_se_squeeze (GlobalAver (None, 1248) 0 ['block6g_activation[0][0]']
agePooling2D)
block6g_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6g_se_squeeze[0][0]']
block6g_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6g_se_reshape[0][0]']
block6g_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6g_se_reduce[0][0]']
block6g_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6g_activation[0][0]',
'block6g_se_expand[0][0]']
block6g_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6g_se_excite[0][0]']
block6g_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6g_project_conv[0][0]']
lization)
block6g_drop (Dropout) (None, 2, 2, 208) 0 ['block6g_project_bn[0][0]']
block6g_add (Add) (None, 2, 2, 208) 0 ['block6g_drop[0][0]',
'block6f_add[0][0]']
block6h_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6g_add[0][0]']
block6h_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6h_expand_conv[0][0]']
ization)
block6h_expand_activation (Act (None, 2, 2, 1248) 0 ['block6h_expand_bn[0][0]']
ivation)
block6h_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6h_expand_activation[0][0]
2D) ']
block6h_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6h_dwconv2[0][0]']
)
block6h_activation (Activation (None, 2, 2, 1248) 0 ['block6h_bn[0][0]']
)
block6h_se_squeeze (GlobalAver (None, 1248) 0 ['block6h_activation[0][0]']
agePooling2D)
block6h_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6h_se_squeeze[0][0]']
block6h_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6h_se_reshape[0][0]']
block6h_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6h_se_reduce[0][0]']
block6h_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6h_activation[0][0]',
'block6h_se_expand[0][0]']
block6h_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6h_se_excite[0][0]']
block6h_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6h_project_conv[0][0]']
lization)
block6h_drop (Dropout) (None, 2, 2, 208) 0 ['block6h_project_bn[0][0]']
block6h_add (Add) (None, 2, 2, 208) 0 ['block6h_drop[0][0]',
'block6g_add[0][0]']
block6i_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6h_add[0][0]']
block6i_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6i_expand_conv[0][0]']
ization)
block6i_expand_activation (Act (None, 2, 2, 1248) 0 ['block6i_expand_bn[0][0]']
ivation)
block6i_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6i_expand_activation[0][0]
2D) ']
block6i_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6i_dwconv2[0][0]']
)
block6i_activation (Activation (None, 2, 2, 1248) 0 ['block6i_bn[0][0]']
)
block6i_se_squeeze (GlobalAver (None, 1248) 0 ['block6i_activation[0][0]']
agePooling2D)
block6i_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6i_se_squeeze[0][0]']
block6i_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6i_se_reshape[0][0]']
block6i_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6i_se_reduce[0][0]']
block6i_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6i_activation[0][0]',
'block6i_se_expand[0][0]']
block6i_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6i_se_excite[0][0]']
block6i_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6i_project_conv[0][0]']
lization)
block6i_drop (Dropout) (None, 2, 2, 208) 0 ['block6i_project_bn[0][0]']
block6i_add (Add) (None, 2, 2, 208) 0 ['block6i_drop[0][0]',
'block6h_add[0][0]']
block6j_expand_conv (Conv2D) (None, 2, 2, 1248) 259584 ['block6i_add[0][0]']
block6j_expand_bn (BatchNormal (None, 2, 2, 1248) 4992 ['block6j_expand_conv[0][0]']
ization)
block6j_expand_activation (Act (None, 2, 2, 1248) 0 ['block6j_expand_bn[0][0]']
ivation)
block6j_dwconv2 (DepthwiseConv (None, 2, 2, 1248) 11232 ['block6j_expand_activation[0][0]
2D) ']
block6j_bn (BatchNormalization (None, 2, 2, 1248) 4992 ['block6j_dwconv2[0][0]']
)
block6j_activation (Activation (None, 2, 2, 1248) 0 ['block6j_bn[0][0]']
)
block6j_se_squeeze (GlobalAver (None, 1248) 0 ['block6j_activation[0][0]']
agePooling2D)
block6j_se_reshape (Reshape) (None, 1, 1, 1248) 0 ['block6j_se_squeeze[0][0]']
block6j_se_reduce (Conv2D) (None, 1, 1, 52) 64948 ['block6j_se_reshape[0][0]']
block6j_se_expand (Conv2D) (None, 1, 1, 1248) 66144 ['block6j_se_reduce[0][0]']
block6j_se_excite (Multiply) (None, 2, 2, 1248) 0 ['block6j_activation[0][0]',
'block6j_se_expand[0][0]']
block6j_project_conv (Conv2D) (None, 2, 2, 208) 259584 ['block6j_se_excite[0][0]']
block6j_project_bn (BatchNorma (None, 2, 2, 208) 832 ['block6j_project_conv[0][0]']
lization)
block6j_drop (Dropout) (None, 2, 2, 208) 0 ['block6j_project_bn[0][0]']
block6j_add (Add) (None, 2, 2, 208) 0 ['block6j_drop[0][0]',
'block6i_add[0][0]']
top_conv (Conv2D) (None, 2, 2, 1408) 292864 ['block6j_add[0][0]']
top_bn (BatchNormalization) (None, 2, 2, 1408) 5632 ['top_conv[0][0]']
top_activation (Activation) (None, 2, 2, 1408) 0 ['top_bn[0][0]']
==================================================================================================
Total params: 8,769,374
Trainable params: 8,687,086
Non-trainable params: 82,288
__________________________________________________________________________________________________
# Model 5: EfficientNet
def model_builder_efficientnet(input_shape, tune=0):
"""Transfer approach with the EfficientNet Model
"""
efficientnet_model = ap.EfficientNetV2B2(include_top=False, weights="imagenet", input_shape=input_shape)
# Making all the layers of the VGG model non-trainable
if tune > 0:
for layer in efficientnet_model.layers[:-tune]:
layer.trainable = False
else:
for layer in efficientnet_model.layers:
layer.trainable = False
model = Sequential([
efficientnet_model,
Flatten(),
Dense(256, activation='relu'),
Dense(128, activation='relu'),
Dropout(0.3),
Dense(64, activation='relu'),
BatchNormalization(),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.001)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_efficientnet, metric_efficientnet, history_efficientnet = model_building_and_evaluating_process(
model_builder_efficientnet, f'Model 5: EfficientNet ({batch_size})', batch_size=batch_size, epochs=20, include_grayscale=False
)
---------------------------------------------
Model 5: EfficientNet (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
efficientnetv2-b2 (Function (None, 2, 2, 1408) 8769374
al)
flatten (Flatten) (None, 5632) 0
dense (Dense) (None, 256) 1442048
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 10,253,090
Trainable params: 1,483,588
Non-trainable params: 8,769,502
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 51s 84ms/step - loss: 1.4558 - accuracy: 0.2583 - val_loss: 1.4238 - val_accuracy: 0.1601 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 38s 79ms/step - loss: 1.4049 - accuracy: 0.2586 - val_loss: 1.3468 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 36s 76ms/step - loss: 1.3985 - accuracy: 0.2543 - val_loss: 1.3631 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 36s 77ms/step - loss: 1.3937 - accuracy: 0.2547 - val_loss: 1.3806 - val_accuracy: 0.2443 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 36s 77ms/step - loss: 1.3930 - accuracy: 0.2603 - val_loss: 1.3744 - val_accuracy: 0.2443 - lr: 0.0010
Epoch 6/20
473/473 [==============================] - 36s 77ms/step - loss: 1.3860 - accuracy: 0.2605 - val_loss: 1.3871 - val_accuracy: 0.2289 - lr: 2.0000e-04
Epoch 7/20
473/473 [==============================] - 38s 79ms/step - loss: 1.3839 - accuracy: 0.2639 - val_loss: 1.3834 - val_accuracy: 0.3679 - lr: 2.0000e-04
1/1 [==============================] - 0s 74ms/step - loss: 1.4044 - accuracy: 0.3125
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
precision recall f1-score support
happy 0.31 1.00 0.48 10
neutral 0.00 0.00 0.00 5
sad 0.00 0.00 0.00 10
surprise 0.00 0.00 0.00 7
accuracy 0.31 32
macro avg 0.08 0.25 0.12 32
weighted avg 0.10 0.31 0.15 32
# Let's try again tuning some of the last layers in the transfer model
batch_size = 32
model, results, history = model_building_and_evaluating_process(model_builder_efficientnet, f'Model 5: Tuned EfficientNet ({batch_size})',
batch_size=batch_size, epochs=20, tune=7, include_grayscale=False)
model_efficientnet.update(model)
metric_efficientnet = pd.concat([metric_efficientnet, results])
history_efficientnet.update(history)
---------------------------------------------
Model 5: Tuned EfficientNet (32) - RGB
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=rgb:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
efficientnetv2-b2 (Function (None, 2, 2, 1408) 8769374
al)
flatten (Flatten) (None, 5632) 0
dense (Dense) (None, 256) 1442048
dense_1 (Dense) (None, 128) 32896
dropout (Dropout) (None, 128) 0
dense_2 (Dense) (None, 64) 8256
batch_normalization (BatchN (None, 64) 256
ormalization)
dense_3 (Dense) (None, 4) 260
=================================================================
Total params: 10,253,090
Trainable params: 2,039,268
Non-trainable params: 8,213,822
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 54s 90ms/step - loss: 1.4554 - accuracy: 0.2542 - val_loss: 1.3559 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 2/20
473/473 [==============================] - 38s 80ms/step - loss: 1.3994 - accuracy: 0.2585 - val_loss: 1.3627 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 3/20
473/473 [==============================] - 38s 81ms/step - loss: 1.3935 - accuracy: 0.2574 - val_loss: 1.3700 - val_accuracy: 0.3667 - lr: 0.0010
Epoch 4/20
473/473 [==============================] - 39s 83ms/step - loss: 1.3933 - accuracy: 0.2640 - val_loss: 1.3663 - val_accuracy: 0.3671 - lr: 0.0010
Epoch 5/20
473/473 [==============================] - 38s 81ms/step - loss: 1.3865 - accuracy: 0.2595 - val_loss: 1.3703 - val_accuracy: 0.2289 - lr: 2.0000e-04
Epoch 6/20
473/473 [==============================] - 39s 82ms/step - loss: 1.3851 - accuracy: 0.2637 - val_loss: 1.3658 - val_accuracy: 0.3667 - lr: 2.0000e-04
1/1 [==============================] - 0s 80ms/step - loss: 1.3944 - accuracy: 0.3125
1/1 [==============================] - 2s 2s/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
precision recall f1-score support
happy 0.31 1.00 0.48 10
neutral 0.00 0.00 0.00 5
sad 0.00 0.00 0.00 10
surprise 0.00 0.00 0.00 7
accuracy 0.31 32
macro avg 0.08 0.25 0.12 32
weighted avg 0.10 0.31 0.15 32
model_name = 'Model 5: EfficientNet'
save_object(history_efficientnet, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_efficientnet, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_efficientnet, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_efficientnet, data_type='Training Set')
loss_and_accuracy_comparisson(history_efficientnet, data_type='Validation Set')
metric_efficientnet.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 5: EfficientNet (32) - RGB | 0.263949 | 0.367892 | 0.312500 |
| Model 5: Tuned EfficientNet (32) - RGB | 0.264015 | 0.367089 | 0.312500 |
Note: You can even go back and build your own architecture on top of the VGG16 Transfer layer and see if you can improve the performance.
⏩ Observations and Insights:
⏩ Recommendations:
Think About It:
# Getting back the history and metrics result during trainning
n = 'Model 3: VGG16'
with open(f'{MODEL_DIR}/{n} - history.pkl', 'rb') as f: history_vgg16 = pickle.load(f)
with open(f'{MODEL_DIR}/{n} - metrics.pkl', 'rb') as f: metric_vgg16 = pickle.load(f)
n = 'Model 4: Resnet V2'
with open(f'{MODEL_DIR}/{n} - history.pkl', 'rb') as f: history_resnet = pickle.load(f)
with open(f'{MODEL_DIR}/{n} - metrics.pkl', 'rb') as f: metric_resnet = pickle.load(f)
n = 'Model 5: EfficientNet'
with open(f'{MODEL_DIR}/{n} - history.pkl', 'rb') as f: history_efficientnet = pickle.load(f)
with open(f'{MODEL_DIR}/{n} - metrics.pkl', 'rb') as f: metric_efficientnet = pickle.load(f)
history_transfer_models = {
'Model 3: Tuned VGG16 (32) - RGB': history_vgg16['Model 3: Tuned VGG16 (32) - RGB'],
'Model 4: Tuned Resnet V2 (32) - RGB': history_resnet['Model 4: Tuned Resnet V2 (32) - RGB'],
'Model 5: Tuned EfficientNet (32) - RGB': history_efficientnet['Model 5: Tuned EfficientNet (32) - RGB']
}
loss_and_accuracy_comparisson(history_transfer_models, data_type='Training Set')
loss_and_accuracy_comparisson(history_transfer_models, data_type='Validation Set')
metric_transfer_models = pd.concat([
metric_vgg16.loc['Model 3: Tuned VGG16 (32) - RGB':],
metric_resnet.loc['Model 4: Tuned Resnet V2 (32) - RGB':],
metric_efficientnet.loc['Model 5: Tuned EfficientNet (32) - RGB':]
])
metric_transfer_models
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 3: Tuned VGG16 (32) - RGB | 0.784367 | 0.712879 | 0.8125 |
| Model 4: Tuned Resnet V2 (32) - RGB | 0.513667 | 0.506731 | 0.3750 |
| Model 5: Tuned EfficientNet (32) - RGB | 0.264015 | 0.367089 | 0.3125 |
_ = metric_transfer_models.plot(kind='bar', figsize=(10,3))
plt.xticks(rotation=0, fontsize=8)
plt.yticks(fontsize=8)
plt.title('Best Performed Trained Transfer Models', fontsize=10)
plt.legend(fontsize=8)
plt.show()
pd.concat([metric_vgg16, metric_resnet, metric_efficientnet]).plot(kind='bar', figsize=(15,3))
plt.xticks(rotation=0, fontsize=8)
plt.yticks(fontsize=8)
plt.title('Trained Transfer Models', fontsize=10)
plt.legend(fontsize=8)
plt.show()
⏩ Observations and Insights:
Now that we have tried multiple pre-trained models, let's build a complex CNN architecture and see if we can get better performance.
In this section, we will build a more complex Convolutional Neural Network Model that has close to as many parameters as we had in our Transfer Learning Models. However, we will have only 1 input channel for our input images.
In this section, we are creating data loaders which we will use as inputs to the more Complicated Convolutional Neural Network. We will go ahead with color_mode = 'grayscale'.
train_generator_gray, _, _ = create_data_loaders(IMG_DIR, 'grayscale')
# Taking a look at some examples of our augmented training data with color_mode='rgb'.
images, labels = next(train_generator_gray)
fig, axes = plt.subplots(1, 8, figsize = (10, 2))
for (image, label, ax) in zip(images, labels, axes.flatten()):
ax.imshow(image, cmap='gray')
ax.set_title(FACE_EXPRESSIONS[list(label).index(1)], fontsize=8)
ax.axis('off')
plt.show()
Data with color_mode=grayscale: Found 15109 images belonging to 4 classes. Found 127 images belonging to 4 classes. Found 4977 images belonging to 4 classes.
# Model 6: Complex Neural Network
def model_builder_complex_cnn(input_shape):
"""Creating a Complex Neural Network.
"""
model = Sequential([
Conv2D(filters=64, kernel_size=2, activation='relu', padding='same', input_shape=input_shape),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=128, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=512, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=512, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Conv2D(filters=128, kernel_size=2, activation='relu', padding='same'),
BatchNormalization(),
LeakyReLU(0.1),
MaxPooling2D(pool_size=2),
Dropout(0.2),
Flatten(),
Dense(256, activation='relu'),
BatchNormalization(),
Dropout(0.4),
Dense(512, activation='relu'),
BatchNormalization(),
Dropout(0.4),
Dense(4, activation='softmax')
])
# Compile model
optimizer = Adam(learning_rate = 0.01) # Using SGD Optimizer
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
print(model.summary())
return model
batch_size = 32
model_complex_cnn, metric_complex_cnn, history_complex_cnn = model_building_and_evaluating_process(
model_builder_complex_cnn, f'Model 6: Complex CNN ({batch_size})', batch_size=batch_size, epochs=20, include_rgb=False
)
---------------------------------------------
Model 6: Complex CNN (32) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
Epoch 1/20
473/473 [==============================] - 44s 83ms/step - loss: 1.7116 - accuracy: 0.2803 - val_loss: 1.5907 - val_accuracy: 0.2536 - lr: 0.0100
Epoch 2/20
473/473 [==============================] - 39s 82ms/step - loss: 1.3762 - accuracy: 0.3332 - val_loss: 1.2800 - val_accuracy: 0.3940 - lr: 0.0100
Epoch 3/20
473/473 [==============================] - 39s 82ms/step - loss: 1.2049 - accuracy: 0.4303 - val_loss: 1.2257 - val_accuracy: 0.4577 - lr: 0.0100
Epoch 4/20
473/473 [==============================] - 39s 82ms/step - loss: 1.0813 - accuracy: 0.5174 - val_loss: 1.0792 - val_accuracy: 0.5248 - lr: 0.0100
Epoch 5/20
473/473 [==============================] - 39s 82ms/step - loss: 1.0102 - accuracy: 0.5609 - val_loss: 0.9836 - val_accuracy: 0.5817 - lr: 0.0100
Epoch 6/20
473/473 [==============================] - 38s 81ms/step - loss: 0.9574 - accuracy: 0.5900 - val_loss: 1.1192 - val_accuracy: 0.5357 - lr: 0.0100
Epoch 7/20
473/473 [==============================] - 38s 80ms/step - loss: 0.9274 - accuracy: 0.6116 - val_loss: 1.1068 - val_accuracy: 0.5043 - lr: 0.0100
Epoch 8/20
473/473 [==============================] - 39s 83ms/step - loss: 0.9071 - accuracy: 0.6203 - val_loss: 0.8265 - val_accuracy: 0.6536 - lr: 0.0100
Epoch 9/20
473/473 [==============================] - 38s 81ms/step - loss: 0.8800 - accuracy: 0.6366 - val_loss: 0.8600 - val_accuracy: 0.6564 - lr: 0.0100
Epoch 10/20
473/473 [==============================] - 38s 81ms/step - loss: 0.8668 - accuracy: 0.6406 - val_loss: 0.8620 - val_accuracy: 0.6323 - lr: 0.0100
Epoch 11/20
473/473 [==============================] - 38s 80ms/step - loss: 0.8432 - accuracy: 0.6469 - val_loss: 3.5923 - val_accuracy: 0.5337 - lr: 0.0100
Epoch 12/20
473/473 [==============================] - 39s 82ms/step - loss: 0.7678 - accuracy: 0.6855 - val_loss: 0.6736 - val_accuracy: 0.7281 - lr: 0.0020
Epoch 13/20
473/473 [==============================] - 38s 81ms/step - loss: 0.7385 - accuracy: 0.7013 - val_loss: 0.6852 - val_accuracy: 0.7231 - lr: 0.0020
Epoch 14/20
473/473 [==============================] - 38s 81ms/step - loss: 0.7189 - accuracy: 0.7120 - val_loss: 0.6652 - val_accuracy: 0.7275 - lr: 0.0020
Epoch 15/20
473/473 [==============================] - 38s 81ms/step - loss: 0.7087 - accuracy: 0.7169 - val_loss: 0.6839 - val_accuracy: 0.7241 - lr: 0.0020
Epoch 16/20
473/473 [==============================] - 39s 82ms/step - loss: 0.6888 - accuracy: 0.7221 - val_loss: 0.6374 - val_accuracy: 0.7434 - lr: 0.0020
Epoch 17/20
473/473 [==============================] - 39s 83ms/step - loss: 0.6760 - accuracy: 0.7274 - val_loss: 0.6516 - val_accuracy: 0.7326 - lr: 0.0020
Epoch 18/20
473/473 [==============================] - 40s 84ms/step - loss: 0.6656 - accuracy: 0.7339 - val_loss: 0.6614 - val_accuracy: 0.7296 - lr: 0.0020
Epoch 19/20
473/473 [==============================] - 40s 84ms/step - loss: 0.6519 - accuracy: 0.7387 - val_loss: 0.6358 - val_accuracy: 0.7472 - lr: 0.0020
Epoch 20/20
473/473 [==============================] - 38s 81ms/step - loss: 0.6468 - accuracy: 0.7395 - val_loss: 0.7484 - val_accuracy: 0.6908 - lr: 0.0020
1/1 [==============================] - 0s 38ms/step - loss: 0.7757 - accuracy: 0.7500
1/1 [==============================] - 0s 199ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 2 3 0 2 3 2 1 0 2 1 3 2 0 3 0 3 2 0 2 2 2 2 2 0 2 1 0 1 2 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.50 0.40 0.44 5
sad 0.64 0.90 0.75 10
surprise 1.00 0.71 0.83 7
accuracy 0.75 32
macro avg 0.76 0.70 0.72 32
weighted avg 0.78 0.75 0.75 32
batch_size = 64
model, results, history = model_building_and_evaluating_process(model_builder_complex_cnn, f'Model 6: Complex CNN ({batch_size})',
batch_size=batch_size, epochs=20, include_rgb=False)
model_complex_cnn.update(model)
metric_complex_cnn = pd.concat([metric_complex_cnn, results])
history_complex_cnn.update(history)
---------------------------------------------
Model 6: Complex CNN (64) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
Epoch 1/20
237/237 [==============================] - 42s 163ms/step - loss: 1.7046 - accuracy: 0.2904 - val_loss: 1.4634 - val_accuracy: 0.2443 - lr: 0.0100
Epoch 2/20
237/237 [==============================] - 38s 159ms/step - loss: 1.4348 - accuracy: 0.3182 - val_loss: 1.3236 - val_accuracy: 0.3430 - lr: 0.0100
Epoch 3/20
237/237 [==============================] - 38s 159ms/step - loss: 1.2923 - accuracy: 0.3857 - val_loss: 1.2863 - val_accuracy: 0.3436 - lr: 0.0100
Epoch 4/20
237/237 [==============================] - 38s 158ms/step - loss: 1.1288 - accuracy: 0.4879 - val_loss: 1.1257 - val_accuracy: 0.5224 - lr: 0.0100
Epoch 5/20
237/237 [==============================] - 37s 156ms/step - loss: 1.0201 - accuracy: 0.5523 - val_loss: 1.1093 - val_accuracy: 0.4999 - lr: 0.0100
Epoch 6/20
237/237 [==============================] - 37s 157ms/step - loss: 0.9889 - accuracy: 0.5671 - val_loss: 1.4645 - val_accuracy: 0.4836 - lr: 0.0100
Epoch 7/20
237/237 [==============================] - 38s 159ms/step - loss: 0.9374 - accuracy: 0.6017 - val_loss: 0.9721 - val_accuracy: 0.5893 - lr: 0.0100
Epoch 8/20
237/237 [==============================] - 37s 156ms/step - loss: 0.9039 - accuracy: 0.6202 - val_loss: 1.1374 - val_accuracy: 0.4933 - lr: 0.0100
Epoch 9/20
237/237 [==============================] - 38s 160ms/step - loss: 0.9106 - accuracy: 0.6167 - val_loss: 0.7781 - val_accuracy: 0.6827 - lr: 0.0100
Epoch 10/20
237/237 [==============================] - 37s 156ms/step - loss: 0.8654 - accuracy: 0.6390 - val_loss: 0.9844 - val_accuracy: 0.5897 - lr: 0.0100
Epoch 11/20
237/237 [==============================] - 37s 156ms/step - loss: 0.8498 - accuracy: 0.6473 - val_loss: 0.9861 - val_accuracy: 0.5586 - lr: 0.0100
Epoch 12/20
237/237 [==============================] - 37s 157ms/step - loss: 0.8299 - accuracy: 0.6605 - val_loss: 0.8312 - val_accuracy: 0.6508 - lr: 0.0100
Epoch 13/20
237/237 [==============================] - 38s 159ms/step - loss: 0.7649 - accuracy: 0.6856 - val_loss: 0.7060 - val_accuracy: 0.7201 - lr: 0.0020
Epoch 14/20
237/237 [==============================] - 38s 159ms/step - loss: 0.7394 - accuracy: 0.6950 - val_loss: 0.6770 - val_accuracy: 0.7306 - lr: 0.0020
Epoch 15/20
237/237 [==============================] - 37s 155ms/step - loss: 0.7169 - accuracy: 0.7086 - val_loss: 0.6951 - val_accuracy: 0.7199 - lr: 0.0020
Epoch 16/20
237/237 [==============================] - 37s 156ms/step - loss: 0.7073 - accuracy: 0.7137 - val_loss: 0.7032 - val_accuracy: 0.7195 - lr: 0.0020
Epoch 17/20
237/237 [==============================] - 37s 156ms/step - loss: 0.6990 - accuracy: 0.7155 - val_loss: 0.7149 - val_accuracy: 0.7145 - lr: 0.0020
Epoch 18/20
237/237 [==============================] - 37s 158ms/step - loss: 0.6783 - accuracy: 0.7285 - val_loss: 0.6455 - val_accuracy: 0.7408 - lr: 4.0000e-04
Epoch 19/20
237/237 [==============================] - 37s 155ms/step - loss: 0.6710 - accuracy: 0.7309 - val_loss: 0.6517 - val_accuracy: 0.7392 - lr: 4.0000e-04
Epoch 20/20
237/237 [==============================] - 38s 158ms/step - loss: 0.6648 - accuracy: 0.7282 - val_loss: 0.6724 - val_accuracy: 0.7410 - lr: 4.0000e-04
1/1 [==============================] - 0s 38ms/step - loss: 0.7054 - accuracy: 0.6875
1/1 [==============================] - 0s 196ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 0 3 2 0 3 0 3 2 0 2 2 2 1 1 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.90 0.90 0.90 10
neutral 0.38 0.60 0.46 5
sad 0.56 0.50 0.53 10
surprise 1.00 0.71 0.83 7
accuracy 0.69 32
macro avg 0.71 0.68 0.68 32
weighted avg 0.73 0.69 0.70 32
batch_size = 128
model, results, history = model_building_and_evaluating_process(model_builder_complex_cnn, f'Model 6: Complex CNN ({batch_size})',
batch_size=batch_size, epochs=20, include_rgb=False)
model_complex_cnn.update(model)
metric_complex_cnn = pd.concat([metric_complex_cnn, results])
history_complex_cnn.update(history)
---------------------------------------------
Model 6: Complex CNN (128) - GRAY
---------------------------------------------
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
Epoch 1/20
119/119 [==============================] - 41s 316ms/step - loss: 1.7298 - accuracy: 0.2822 - val_loss: 1.5841 - val_accuracy: 0.3617 - lr: 0.0100
Epoch 2/20
119/119 [==============================] - 37s 309ms/step - loss: 1.5195 - accuracy: 0.2928 - val_loss: 2.0473 - val_accuracy: 0.2849 - lr: 0.0100
Epoch 3/20
119/119 [==============================] - 37s 311ms/step - loss: 1.4196 - accuracy: 0.3137 - val_loss: 1.3419 - val_accuracy: 0.3749 - lr: 0.0100
Epoch 4/20
119/119 [==============================] - 37s 308ms/step - loss: 1.3748 - accuracy: 0.3305 - val_loss: 1.4027 - val_accuracy: 0.3669 - lr: 0.0100
Epoch 5/20
119/119 [==============================] - 38s 316ms/step - loss: 1.3493 - accuracy: 0.3386 - val_loss: 1.3150 - val_accuracy: 0.3783 - lr: 0.0100
Epoch 6/20
119/119 [==============================] - 38s 315ms/step - loss: 1.3065 - accuracy: 0.3704 - val_loss: 1.2485 - val_accuracy: 0.4137 - lr: 0.0100
Epoch 7/20
119/119 [==============================] - 38s 315ms/step - loss: 1.2580 - accuracy: 0.4076 - val_loss: 1.2150 - val_accuracy: 0.4505 - lr: 0.0100
Epoch 8/20
119/119 [==============================] - 38s 315ms/step - loss: 1.1512 - accuracy: 0.4814 - val_loss: 1.1007 - val_accuracy: 0.5101 - lr: 0.0100
Epoch 9/20
119/119 [==============================] - 38s 315ms/step - loss: 1.0675 - accuracy: 0.5256 - val_loss: 1.0095 - val_accuracy: 0.5660 - lr: 0.0100
Epoch 10/20
119/119 [==============================] - 38s 315ms/step - loss: 0.9822 - accuracy: 0.5731 - val_loss: 1.0215 - val_accuracy: 0.5769 - lr: 0.0100
Epoch 11/20
119/119 [==============================] - 37s 311ms/step - loss: 0.9774 - accuracy: 0.5779 - val_loss: 1.3275 - val_accuracy: 0.4762 - lr: 0.0100
Epoch 12/20
119/119 [==============================] - 37s 310ms/step - loss: 0.9699 - accuracy: 0.5875 - val_loss: 1.1155 - val_accuracy: 0.5634 - lr: 0.0100
Epoch 13/20
119/119 [==============================] - 38s 315ms/step - loss: 0.8904 - accuracy: 0.6245 - val_loss: 0.8109 - val_accuracy: 0.6649 - lr: 0.0020
Epoch 14/20
119/119 [==============================] - 37s 314ms/step - loss: 0.8687 - accuracy: 0.6342 - val_loss: 0.7978 - val_accuracy: 0.6701 - lr: 0.0020
Epoch 15/20
119/119 [==============================] - 38s 316ms/step - loss: 0.8469 - accuracy: 0.6472 - val_loss: 0.7810 - val_accuracy: 0.6751 - lr: 0.0020
Epoch 16/20
119/119 [==============================] - 37s 313ms/step - loss: 0.8385 - accuracy: 0.6521 - val_loss: 1.3783 - val_accuracy: 0.3994 - lr: 0.0020
Epoch 17/20
119/119 [==============================] - 38s 316ms/step - loss: 0.8340 - accuracy: 0.6547 - val_loss: 0.7525 - val_accuracy: 0.6906 - lr: 0.0020
Epoch 18/20
119/119 [==============================] - 37s 310ms/step - loss: 0.8155 - accuracy: 0.6619 - val_loss: 0.8422 - val_accuracy: 0.6701 - lr: 0.0020
Epoch 19/20
119/119 [==============================] - 37s 310ms/step - loss: 0.8082 - accuracy: 0.6675 - val_loss: 0.7609 - val_accuracy: 0.6866 - lr: 0.0020
Epoch 20/20
119/119 [==============================] - 37s 313ms/step - loss: 0.7919 - accuracy: 0.6761 - val_loss: 0.7667 - val_accuracy: 0.6878 - lr: 0.0020
1/1 [==============================] - 0s 37ms/step - loss: 0.8769 - accuracy: 0.5938
1/1 [==============================] - 0s 195ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 2 3 2 0 3 0 3 2 2 2 2 2 1 1 0 2 2 0 0 1 0 2]
precision recall f1-score support
happy 0.78 0.70 0.74 10
neutral 0.33 0.40 0.36 5
sad 0.42 0.50 0.45 10
surprise 1.00 0.71 0.83 7
accuracy 0.59 32
macro avg 0.63 0.58 0.60 32
weighted avg 0.64 0.59 0.61 32
model_name = 'Model 6: Complex CNN'
save_object(history_complex_cnn, f'{MODEL_DIR}/{model_name} - history.pkl')
save_object(metric_complex_cnn, f'{MODEL_DIR}/{model_name} - metrics.pkl')
save_object(model_complex_cnn, f'{MODEL_DIR}/{model_name} - models.pkl')
loss_and_accuracy_comparisson(history_complex_cnn, data_type='Training Set')
loss_and_accuracy_comparisson(history_complex_cnn, data_type='Validation Set')
metric_complex_cnn.style.highlight_max(color = "lightgreen", axis = 0)
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 6: Complex CNN (32) - GRAY | 0.738699 | 0.747237 | 0.750000 |
| Model 6: Complex CNN (64) - GRAY | 0.728175 | 0.741009 | 0.687500 |
| Model 6: Complex CNN (128) - GRAY | 0.654709 | 0.690577 | 0.593750 |
⏩ Observations and Insights:
⏩ Recommendations:
# Getting back the history and metrics result during trainning
with open(f'{MODEL_DIR}/Model 1: Base Neural Network - metrics.pkl', 'rb') as f: metric_base_cnn = pickle.load(f)
with open(f'{MODEL_DIR}/Model 2: Enhanced Neural Network - metrics.pkl', 'rb') as f: metric_enhanced_cnn = pickle.load(f)
with open(f'{MODEL_DIR}/Model 3: VGG16 - metrics.pkl', 'rb') as f: metric_vgg16 = pickle.load(f)
with open(f'{MODEL_DIR}/Model 4: Resnet V2 - metrics.pkl', 'rb') as f: metric_resnet = pickle.load(f)
with open(f'{MODEL_DIR}/Model 5: EfficientNet - metrics.pkl', 'rb') as f: metric_efficientnet = pickle.load(f)
with open(f'{MODEL_DIR}/Model 6: Complex CNN - metrics.pkl', 'rb') as f: metric_complex_cnn = pickle.load(f)
best_performed_models = pd.concat([
metric_base_cnn.loc['Model 1: Base Neural Network (32) - RGB':'Model 1: Base Neural Network (32) - RGB'],
metric_enhanced_cnn['Model 2: Enhanced Neural Network (64) - RGB':'Model 2: Enhanced Neural Network (64) - RGB'],
metric_vgg16.loc['Model 3: Tuned VGG16 (32) - RGB':'Model 3: Tuned VGG16 (32) - RGB'],
metric_resnet.loc['Model 4: Tuned Resnet V2 (32) - RGB':'Model 4: Tuned Resnet V2 (32) - RGB'],
metric_efficientnet.loc['Model 5: EfficientNet (32) - RGB':'Model 5: EfficientNet (32) - RGB'],
metric_complex_cnn.loc['Model 6: Complex CNN (32) - GRAY':'Model 6: Complex CNN (32) - GRAY']
])
best_performed_models
| Accuracy Train | Accuracy Val | Accuracy Test | |
|---|---|---|---|
| Model 1: Base Neural Network (32) - RGB | 0.705871 | 0.719108 | 0.78125 |
| Model 2: Enhanced Neural Network (64) - RGB | 0.826593 | 0.736789 | 0.68750 |
| Model 3: Tuned VGG16 (32) - RGB | 0.784367 | 0.712879 | 0.81250 |
| Model 4: Tuned Resnet V2 (32) - RGB | 0.513667 | 0.506731 | 0.37500 |
| Model 5: EfficientNet (32) - RGB | 0.263949 | 0.367892 | 0.31250 |
| Model 6: Complex CNN (32) - GRAY | 0.738699 | 0.747237 | 0.75000 |
_ = best_performed_models.plot(kind='bar', figsize=(20,3))
plt.xticks(rotation=0, fontsize=8)
plt.axhline(y=0.74, lw=1, ls='--', c='red')
plt.yticks(fontsize=6)
plt.title('Best Performed Models', fontsize=10)
plt.legend(fontsize=8)
plt.show()
# Getting back a model
# Preparing the environment
init_environment()
_, test_generator, _ = create_data_loaders(IMG_DIR, 'grayscale')
X_test, y_test = test_generator.next()
# Loading the model
model = model_builder_complex_cnn(input_shape=(48, 48, 1))
model.load_weights(f'{MODEL_DIR}/Model 6: Complex CNN (32) - GRAY.h5')
# Re-evaluate the model
loss, acc = model.evaluate(X_test, y_test, verbose=2)
model.evaluate(X_test, y_test, verbose=1)
print("Restored model, accuracy: {:5.2f}%".format(100 * acc))
Initializing TF Session and Seed
Data with color_mode=grayscale:
Found 15109 images belonging to 4 classes.
Found 127 images belonging to 4 classes.
Found 4977 images belonging to 4 classes.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 48, 48, 64) 320
batch_normalization (BatchN (None, 48, 48, 64) 256
ormalization)
leaky_re_lu (LeakyReLU) (None, 48, 48, 64) 0
max_pooling2d (MaxPooling2D (None, 24, 24, 64) 0
)
dropout (Dropout) (None, 24, 24, 64) 0
conv2d_1 (Conv2D) (None, 24, 24, 128) 32896
batch_normalization_1 (Batc (None, 24, 24, 128) 512
hNormalization)
leaky_re_lu_1 (LeakyReLU) (None, 24, 24, 128) 0
max_pooling2d_1 (MaxPooling (None, 12, 12, 128) 0
2D)
dropout_1 (Dropout) (None, 12, 12, 128) 0
conv2d_2 (Conv2D) (None, 12, 12, 512) 262656
batch_normalization_2 (Batc (None, 12, 12, 512) 2048
hNormalization)
leaky_re_lu_2 (LeakyReLU) (None, 12, 12, 512) 0
max_pooling2d_2 (MaxPooling (None, 6, 6, 512) 0
2D)
dropout_2 (Dropout) (None, 6, 6, 512) 0
conv2d_3 (Conv2D) (None, 6, 6, 512) 1049088
batch_normalization_3 (Batc (None, 6, 6, 512) 2048
hNormalization)
leaky_re_lu_3 (LeakyReLU) (None, 6, 6, 512) 0
max_pooling2d_3 (MaxPooling (None, 3, 3, 512) 0
2D)
dropout_3 (Dropout) (None, 3, 3, 512) 0
conv2d_4 (Conv2D) (None, 3, 3, 128) 262272
batch_normalization_4 (Batc (None, 3, 3, 128) 512
hNormalization)
leaky_re_lu_4 (LeakyReLU) (None, 3, 3, 128) 0
max_pooling2d_4 (MaxPooling (None, 1, 1, 128) 0
2D)
dropout_4 (Dropout) (None, 1, 1, 128) 0
flatten (Flatten) (None, 128) 0
dense (Dense) (None, 256) 33024
batch_normalization_5 (Batc (None, 256) 1024
hNormalization)
dropout_5 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 512) 131584
batch_normalization_6 (Batc (None, 512) 2048
hNormalization)
dropout_6 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 4) 2052
=================================================================
Total params: 1,782,340
Trainable params: 1,778,116
Non-trainable params: 4,224
_________________________________________________________________
None
1/1 - 0s - loss: 0.7360 - accuracy: 0.7500 - 495ms/epoch - 495ms/step
1/1 [==============================] - 0s 40ms/step - loss: 0.7360 - accuracy: 0.7500
Restored model, accuracy: 75.00%
# Getting the history back as this a required parameter in our defined function.
with open(f'{MODEL_DIR}/Model 6: Complex CNN - history.pkl', 'rb') as f: history = pickle.load(f)
# Preparing the variables
y_pred = np.argmax(model.predict(X_test), axis=1)
y_test = np.argmax(y_test, axis=1)
print('Actual :', y_test)
print('Prediction:', y_pred)
print('\n\n')
# Plotting the classification report and confussion matrix
results = metrics_score(y_test, y_pred, 'Model 6: Complex CNN (32) - GRAY', history['Model 6: Complex CNN (32) - GRAY'])
1/1 [==============================] - 0s 216ms/step
Actual : [0 1 3 0 1 3 2 1 0 2 0 3 2 2 3 0 3 2 0 2 1 3 2 2 0 0 3 0 1 2 0 2]
Prediction: [0 1 3 0 2 3 2 1 0 1 1 3 2 0 3 0 3 2 0 2 1 3 1 2 0 2 1 0 1 1 0 2]
precision recall f1-score support
happy 0.89 0.80 0.84 10
neutral 0.44 0.80 0.57 5
sad 0.75 0.60 0.67 10
surprise 1.00 0.86 0.92 7
accuracy 0.75 32
macro avg 0.77 0.76 0.75 32
weighted avg 0.80 0.75 0.76 32
⏩ Observations and Insights:
⏩ Observations and Insights:
A couple of things can be done to try to improve this model:
⏩ Conclusions and Insights:
What are the most meaningful insights from the data relevant to the problem?
⏩ Even do, the dataset is balanced, features in some of the classes are difficult to identify, even for the human eyes. A review of the training dataset is necessary to confirm data is good enough. As it was mentioned before some images seem to belong to another class that the one they are currently on.
⏩ With the exception of the "Model 4: Tuned Resnet V2" and "Model 5: Tuned EfficientNet", all models achieve more than 70% of accuracy. However, the time to tune and explore each model was huge.
⏩ "Model 4: Tuned Resnet V2" and "Model 5: Tuned EfficientNet" need to be recompiled with the "tune" parameter increased to enable a longer backpropagation, exploring if this improves the model.
⏩ There are 3 models in the final solution that can be selected:
- "Model 2: Enhanced Neural Network (64) - RGB" that achieved 73.7% of accuracy.
- "Model 3: Tuned VGG16 (32) - RGB" that achieved 71.3% of accuracy, had a consistent evolution through epochs during the training, and took advantage of the VGG16 models.
- "Model 6: Complex CNN (32) - GRAY" that achieved 74.7% of accuracy, and had a good performance on the testing set.
⏩ With the current trained models and results, I recommend "Model 6: Complex CNN (32) - GRAY" as it achieved the best validation performance.